ffxiv-craft-opt-web icon indicating copy to clipboard operation
ffxiv-craft-opt-web copied to clipboard

Bug with higher level recipe penalty calculation for crafter levels<50 (effective level<120) results in wrong progress value

Open danj2k opened this issue 8 years ago • 61 comments

~~I'm expecting this is not a bug, just something I am doing wrong,~~ as I only just discovered this website. EDIT: I now believe it is a bug, see my subsequent comment below for details.

What I am seeing is that after I am inputting my level, stats etcetera, choosing a recipe and then starting the simulator, the output contains more synthesis steps than it should, and looking at the action logs it seems that this is because the progress values it is using do not match what I am seeing in-game.

The specific example I was currently trying is a level 35 Culinarian making level 40 Apple Juice. I have 164 craftsmanship, 159 control and 255 CP. I also have Careful Synthesis, Hasty Touch and Manipulation as cross-class actions.

Here's the log of a result I just got:

Seed: 1499178899141, Use Conditions: true

Monte Carlo Example
===================
 # Action                           DUR    CP      QUA      PRG    IQ  WWYW   CTL  QINC  BPRG  BQUA   WAC       Cond   S/F
 0                                   80   255        0        0     0     0   159     0     0     0     0     Normal     0
 1 Inner Quiet                       80   237        0        0     0     0   159     0     0     0     0     Normal     1
 2 Hasty Touch                       70   237        0        0     0     0   159     0     0    68     0     Normal     0
 3 Manipulation                      70   149        0        0     0     0   159     0     0     0     0     Normal     1
 4 Careful Synthesis                 70   149        0       16     0     0   159     0    16     0     0     Normal     1
 5 Steady Hand                       80   127        0       16     0     0   159     0     0     0     0     Normal     1
 6 Basic Touch                       80   109       68       16     1     0   159    68     0    68     0     Normal     1
 7 Basic Touch                       70    91      145       16     2     0   191    77     0    77     0       Good     1
 8 Hasty Touch                       60    91      273       16     3     0   223   128     0    85     0     Normal     1
 9 Hasty Touch                       50    91      367       16     4     0   254    94     0    94     0     Normal     1
10 Basic Synthesis                   40    91      367       34     4     0   286     0    18     0     0     Normal     1
11 Manipulation                      40     3      367       34     4     0   286     0     0     0     0     Normal     1
12 Careful Synthesis                 40     3      367       50     4     0   286     0    16     0     0       Good     1
13 Careful Synthesis                 40     3      367       66     4     0   286     0    16     0     0     Normal     1
14 Careful Synthesis                 40     3      367       82     4     0   286     0    16     0     0     Normal     1
15 Careful Synthesis                 30     3      367       98     4     0   286     0    16     0     0       Good     1
16 Careful Synthesis                 20     3      367      114     4     0   286     0    16     0     0     Normal     1
17 Careful Synthesis                 10     3      367      130     4     0   286     0    16     0     0     Normal     1
18 Careful Synthesis                  0     3      367      146     4     0   286     0    16     0     0     Normal     1
Progress Check: true, Durability Check: true, CP Check: true, Tricks Check: true, Reliability Check: true, Cross Class Skills: 2, Wasted Actions: 0

There are way too many synthesis actions in this, and this seems to be because it thinks the progress of Careful Synthesis on this recipe is 16. This does not match what I see in-game, which in fact is 28.

~~So, what have I done wrong? For example, when inputting the stats, should I have input base stats without any gear on, and then input the gear bonuses in the extra boxes on the simulator page?~~ EDIT: I think this is a bug, see below for explanation.

danj2k avatar Jul 04 '17 14:07 danj2k

Oh hey, I think I figured out the problem.

This code in ffxivcraftmodel.js in the section concerning crafterLevel < 120:

        if ((levelDifference < -5)) {
            levelCorrectionFactor = 0.0501 * levelDifference;
        }
        else if ((-5 <= levelDifference) && (levelDifference < 0)) {
            levelCorrectionFactor = 0.10 * levelDifference;
        }

This is applying a greater penalty than is actually present in the game for a -5 level difference (recipe 5 levels above my crafter level). The top formula of the two calculates it correctly.

Showing working:

levelDifference = -5, craftsmanship = 164

(1 + 0.0501 * levelDifference) * (0.214959 * craftsmanship + 1.6)

=(1 + 0.0501 * -5) * (0.214959 * 164 + 1.6)

=(1 - 0.2505) * (35.253276 + 1.6)

=0.7495 * 36.853276

=27.621530362

round(27.621530362) = 28

I think there's something a bit more wrong with this code though, it doesn't make sense to me that less difficult (i.e. between 0 and -5) recipes would have a higher penalty, so fixing this issue is going to be more than just changing some operators.

EDIT: Tried to fix this in-browser using the developer console but this had unexpected results.

EDIT 2: I suggest this change? Is the penalty 10% for recipe progress between 6 and 9 levels above your crafter level, when crafter level is less than 120? That would make more sense.

        if ((levelDifference >= -5)  && (levelDifference < 0)) {
            levelCorrectionFactor = 0.0501 * levelDifference;
        }
        else if ((-5 >= levelDifference)) {
            levelCorrectionFactor = 0.10 * levelDifference;
        }

        levelCorrectedProgress = Math.round((1 + levelCorrectionFactor) * baseProgress);

danj2k avatar Jul 04 '17 19:07 danj2k

Is this specific to Careful Synthesis or are the other synthesis actions producing progress values that don't match the game?

doxxx avatar Jul 04 '17 21:07 doxxx

Given the crafting data we have received in the past, in your situation we would expect a Basic Synthesis to produce 21 progress, which would mean that Careful Synthesis should produce ~18. While this is not quite the same as the Simulator currently shows, it's close enough. So I'm very puzzled by the large difference that you're seeing.

Did you account for any food effects? You can enter these in the bonus attributes section on the right of the simulator status.

doxxx avatar Jul 04 '17 22:07 doxxx

You guys are aware that the formulas changed drastically in 4.0?

From the patch notes: When a recipe's level is higher than your crafter level, the difficulty of raising quality will increase, while the difficulty of raising progress will decrease.

I've also noticed smaller changes across the board but that's the biggest and the only one in the notes.

Basically, all the formulas and stuff you came up with before 4.0 are going to need tweaking.

Thortok avatar Jul 05 '17 04:07 Thortok

I'm now CUL 36 with craftsmanship 168 and control 170. With the same recipe I am now seeing 30 progress per Careful Synthesis, with no food effects.

Screenshot - see the bottom left where it is showing the progress values, and see the top where there are no food icons active (the ones that are active are Free Company actions, but they are just regarding EXP and should have no effect on crafting progress).

Showing working:

levelDifference = -4, craftsmanship = 168

(1 + 0.0501 * levelDifference) * (0.214959 * craftsmanship + 1.6)

=(1 + 0.0501 * -4) * (0.214959 * 168 + 1.6)

=(1 - 0.2004) * (36.113112 + 1.6)

=0.7996 * 37.713112

=30.1554043552

round(30.1554043552) = 30

EDIT: Though I guess since this is Careful Synthesis, your formula needs to be giving a result that's (1/0.9) multiplied by the above, so I guess there's a more fundamental change needed to fix this issue.

danj2k avatar Jul 05 '17 05:07 danj2k

Here's some data I collected this morning:

CUL 36, craftsmanship 168

Level 40 Apple Juice, progress with Basic Synthesis is 33 Level 39 Raptor Stew, progress with Basic Synthesis is 34 Level 38 Knight's Bread, progress with Basic Synthesis is 35 Level 37 Ratatouille, progress with Basic Synthesis is 36 Level 36 Cream Cheese, progress with Basic Synthesis is 37

I'll collect more data with other classes, which will have different craftsmanship due to being different levels.

danj2k avatar Jul 05 '17 16:07 danj2k

I have weaver at 62 and the rest in the mid 50s, and am available to pull whatever information may be useful.

crossedxd avatar Jul 05 '17 21:07 crossedxd

More data:

GSM 25, craftsmanship 130

Level 30 Cut Stone, progress with Basic Synthesis is 25

GSM 26, craftsmanship 130

Level 26 Horn Necklace, progress with Basic Synthesis is 29

ALC 25, craftsmanship 137

Level 29 Linseed Oil, progress with Basic Synthesis is 27 Level 25 Natron, progress with Basic Synthesis is 30

I'm working on the math at the moment to try and figure out what the updated formula needs to be. Assuming that the structure of the formula is the same (because if it isn't it'll take a lot more work to figure out what it should be) then we just need to figure out the coefficients and whatnot, solving:

(1 + a * levelDifference) * (b * craftsmanship + c) ≅ progress

for a, b and c. The difficulty here is the progress numbers we see are rounded from what the formula outputs.

danj2k avatar Jul 05 '17 21:07 danj2k

We used an Excel spreadsheet with initially our own data when we were still playing FFXIV and later with data submitted by helpful players, and ran Excel's solver on it to determine the coefficients. The game numbers being rounded is a significant hindrance to improving the accuracy.

It requires a lot of data and time to nail down the formula accurately enough to be useful. Neither of us have the free time or the game access to do this work and verify its accuracy. Updating recipe lists is mostly automated, which is why I have been able to (mostly) keep up with that.

If somebody can figure out the formula, I'll gladly update the code to use it. @Ermad is doing some great work in PR #139 to incorporate the various new actions and effects.

doxxx avatar Jul 05 '17 23:07 doxxx

Any chance you still have the Excel spreadsheet somewhere, or could give me an idea/template of what I need to pull together to at least kick off the solver process? I'd gladly do the grunt work involved to get the tool back on its feet, considering how much I've leveraged it since I started playing again.

crossedxd avatar Jul 06 '17 02:07 crossedxd

@doxxx I'd also be interested in how that Excel spreadsheet works, I've not used Excel's solver before - I was actually planning on just doing the math by hand - so that would speed up the process.

danj2k avatar Jul 06 '17 05:07 danj2k

@crossedxd to refine the formula to fix this specific issue more data is needed:

Crafting class and level Recipe name and level Craftsmanship stat (don't forget this changes depending what equipment you have on) Amount of progress per Basic Synthesis (ensure you aren't affected by any Food bonuses, or any other bonuses to progress that might be granted by other abilities)

I managed to brute force a possible range of values that can be plugged in to the formula but the more 4.0 data that's available, the, more this can be narrowed down until I can reach a single set of values.

Including recipes that are the same level as you is useful, as this gives a levelDifference of zero which effectively eliminates the first term of the formula.

Including a variety of recipes that have different level differences and different craftsmanship stats is also useful.

danj2k avatar Jul 06 '17 14:07 danj2k

@danj2k I've posted a link to the spreadsheet on the Wiki.

lokyst avatar Jul 06 '17 18:07 lokyst

@lokyst Thanks! Though looking at it I have to say it is considerably over my head.

danj2k avatar Jul 06 '17 18:07 danj2k

Gist of it is to use the inputs (craftsmanship, level etc.) to calculate the progress based on your formula, compare that to the actual result, and then use the Excel solver to minimize the sum of the squared errors by allowing it to change the coefficients of your formula (I've typically put these near the top right of the tab -- you may need to scroll right a lot). Ideally you want the errors to be under 1 point, but because the numbers are truncated this is not always possible.

Squeenix had different formulas for recipes below your level, even level and higher level. In addition, the formulas varied between ARR recipes and Heavensward -- hence all the tabs.

Adding ingenuity into the mix should only be considered once you nail down the base formula(s).

lokyst avatar Jul 06 '17 18:07 lokyst

I ran some tests this evening to pull a spread of progression numbers. All synth actions were performed by a Lv.62 weaver, using the Ruby Cotton Yarn recipe (also Lv.62).

Craftmanship C. Synth Basic Synth C. Synth 2 Standard Synth
356 64 72 86 108
364 66 73 88 110
365 66 73 88 110
373 68 75 90 113
387 70 78 94 117
395 72 80 96 120
396 72 80 96 120
403 73 81 98 122
404 73 81 98 122
411 75 83 100 125
412 75 83 100 125
420 76 85 102 127
434 79 88 105 132
442 80 89 107 134
443 80 89 107 134
444 81 90 108 135
451 82 91 109 137
452 82 91 109 137
453 82 92 110 138
461 84 93 112 140
475 86 96 115 144
483 88 98 117 147
484 88 98 118 147
491 90 100 120 150
492 90 100 120 150
499 91 101 122 152
500 91 101 122 152
508 93 103 124 155
522 95 106 127 159
530 97 108 129 162
531 97 108 130 162
539 99 110 132 165

crossedxd avatar Jul 06 '17 22:07 crossedxd

Thanks, I was able to get the base progress formula for Stormblood recipes from that.

Biggest thing now is to gather data for recipes of a different level then your crafter level.

Ermad avatar Jul 06 '17 23:07 Ermad

I can run more tests if needed, though the cheaper the better. Is there anything specific you would like me to test?

crossedxd avatar Jul 06 '17 23:07 crossedxd

Just need data for as many level differences as possible (both above and below). Don't need as many data points per level difference though, just 2 different craftsmanship values with those 4 actions should suffice.

Ermad avatar Jul 06 '17 23:07 Ermad

I'll work on pulling that now, then. The ridiculous number of data points above was just because I was attempting my own regression work with it, but I don't have any real stats experience, so I'd rather leave that to those who do. :)

crossedxd avatar Jul 06 '17 23:07 crossedxd

Alright, here's what I've managed to pull together so far. Note that this was again accomplished using a Lv.62 weaver. I don't have access to anything above Lv.65 so far, so unfortunately that's as high as I can go.

Recipe Name Recipe Level iLvl (ffxivcrafting) Craftsmanship C. Synth Basic Synth C. Synth 2 Standard Synth
Ramie Thread 54 133 539 155 172 (complete) (complete)
Hallowed Ramie Cloth 56 139 539 155 172 207 (complete)
Crawler Silk 57 142 539 155 172 206 (complete)
Chimerical Felt 58 145 539 155 172 206 (complete)
Chimerical Felt Chausses of Healing 59 145 539 154 172 206 258
Bloodhempen Yarn 61 255 539 134 148 178 223
Ruby Cotton Yarn 62 265 539 99 110 132 165
Ruby Cotton Cap 63 270 539 86 95 114 143
Kudzu Thread 64 273 539 79 87 105 131
Kudzu Cap of Crafting 65 276 539 74 82 98 123
Ramie Thread 54 133 309 88 98 117 147
Hallowed Ramie Cloth 56 139 309 88 98 117 147
Crawler Silk 57 142 309 88 97 117 146
Chimerical Felt 58 145 309 88 97 117 146
Chimerical Felt Chausses of Healing 59 145 309 87 97 117 146
Bloodhempen Yarn 61 255 309 76 84 101 126
Ruby Cotton Yarn 62 265 309 56 62 75 93
Ruby Cotton Cap 63 270 309 48 54 65 81
Kudzu Thread 64 273 309 44 49 59 74
Kudzu Cap of Crafting 65 276 309 42 46 56 70

crossedxd avatar Jul 07 '17 00:07 crossedxd

Thanks. Would it be possible to get values for recipes even higher level difference, or is that the max you are able to craft?

Ermad avatar Jul 07 '17 02:07 Ermad

65 is my current max with weaver at 62. I can grind a bit tomorrow and see if I can unlock the other recipes on weaver, but if something in the 50-60 range is acceptable I can just look into getting the data from one of my other classes.

crossedxd avatar Jul 07 '17 02:07 crossedxd

I pushed an update to my branch (in #139), that has this taken into account. It should be decently accurate now, besides recipes above your level.

Now need more different level difference data points for recipes above your level. Have -5, -8, and -11 from your data so far. All my crafters are level 70, so I am only able to get data for -10 (1 star recipes), which I will do tomorrow.

Also need to check the Ingenuity 1 & II values. The recipes ilvl scaled in same exact fashion from 61-70 as they did from 51-60 in Heavensward. So currently it just scaled the Ingenuity values by the same amount, need to do testing if this is actually accurate.

Ermad avatar Jul 07 '17 03:07 Ermad

Also need data for quality increase when using touches on recipes above your level. My crafters are all 70 so I can only get it for -10 level difference.

Ermad avatar Jul 07 '17 03:07 Ermad

Here are my current class levels (some may be close to leveling up): ALC - 53 ARM - 52 BSM - 52 CRP - 51 CUL - 52 GSM - 54 LTW - 52 WVR - 62

If you have any ideas of what specifically you'd like me to pull data-wise, let me know. I can also look into it a bit tomorrow after work.

crossedxd avatar Jul 07 '17 03:07 crossedxd

Once you get to Weaver to 63 it will open up more. Doing recipes for the 3 levels above you will give -3, -6, -9 then.

Ermad avatar Jul 07 '17 03:07 Ermad

I'll do that tomorrow, then. Material costs may be an issue, but I'll cross that bridge when I come to it.

crossedxd avatar Jul 07 '17 03:07 crossedxd

Thanks a ton! If you were on Cactuar I could help out, but doubt that you are.

The quality increase formula from Heavensward actually stills seems accurate (within a couple points) for recipes at or below your level. The penalty for recipes above your level has changed though, if you are able to record the quality values of a few touches at the same time.

Ermad avatar Jul 07 '17 03:07 Ermad

I'll grab that info as well. Right now I just run a pretty simple info-gathering macro against each recipe (Reclaim, SH II, the 4x synths, and spam synth till I fail). I can just do the same for the touches when I'm info-gathering tomorrow (basic, standard, advanced).

crossedxd avatar Jul 07 '17 03:07 crossedxd

Sounds like @Ermad and @crossedxd have crafter level 50+/60+ well in hand, so I will focus on crafter levels <50 since that's where I'm at:

31 Carpenter 29 Blacksmith n/a Armorer 26 Goldsmith 37 Leatherworker 32 Weaver 26 Alchemist 36 Culinarian

(I will pick up Armorer, it's just I haven't really needed it for anything until now).

danj2k avatar Jul 07 '17 05:07 danj2k

Was doing some testing on it myself, it seems like the higher recipe penalty is just -2.5% on progress for each recipe level capped at 25%, and quality is -5% for each recipe level capped at -50%.

Ingenuity (I & II) seem to follow the same pattern as in Heavensward, so I am happy with their current values unless someone can show some conflicting data.

Ermad avatar Jul 07 '17 14:07 Ermad

So what does that mean from a data collection standpoint? Do you still need me to pull the synth/touch data we were discussing earlier, or would it make more sense for me to just test the solver out and see how it compares to what I see in-game?

crossedxd avatar Jul 07 '17 16:07 crossedxd

Having data for the other level difference values would be good to confirm it, but if its a lot of trouble it should be fine without it.

Would also be nice to get some Heavensward data (crafter level 51-60) to see if that changed (both equal level and below recipe level), while you level those up. Or at least can you see if the values in simulator or still accurate for those.

Ermad avatar Jul 07 '17 16:07 Ermad

What I'll do when I get home is just put together a list of stuff to test and just do it all at once. Right now I'm planning on running synth/touch tests for Lv.64-70 recipes with my Lv.63 weaver, and then run a -10/+10 spread for synth/touch on my Lv.54 goldsmith (or go as high as I can; is 10 too much of a delta?). Let me know if that should suffice, or if there's anything else that would help out. (This data will look like the 2nd table I posted in this thread.)

crossedxd avatar Jul 07 '17 16:07 crossedxd

That looks good to me, thanks again.

Ermad avatar Jul 07 '17 17:07 Ermad

I picked up Armorer and gathered some data this evening, but I'm not having any luck figuring out the constants for recipes below crafter level (where crafter level is < 50).

crafterClass crafterLevel recipeName recipeLevel levelDifference craftsmanship progress
ARM 1 Bronze Ingot 1 0 39 10
ARM 2 Bronze Ingot 1 1 39 10
ARM 2 Bronze Rings 2 0 39 10
ARM 3 Bronze Ingot 1 2 39 11
ARM 3 Bronze Plate 3 0 39 10
ARM 4 Bronze Hoplon 5 -1 39 9
ARM 6 Bronze Hoplon 5 1 39 10
ARM 6 Bronze Rivets 2 4 65 18
ARM 6 Bronze Alembic 3 3 65 18
ARM 7 Bronze Ingot 1 6 65 19
ARM 7 Bronze Rings 2 5 65 19
ARM 7 Bronze Chain Coif 8 -1 65 15
ARM 8 Bronze Ingot 1 7 65 20

danj2k avatar Jul 07 '17 17:07 danj2k

Hard to do much data that has that little craftsmanship, since so much information is lost by the rounding. Would be better to get as much data as you can from your level 30s/40s crafter instead.

Ermad avatar Jul 07 '17 18:07 Ermad

So at Lv.63, it appears as though I only have recipes up to 68 available to me. Do we really need anything greater than a +5/-5 split for testing purposes?

crossedxd avatar Jul 07 '17 22:07 crossedxd

That should be good, I think its capped at a penalty for 10 recipe levels above and +5 is 15 recipe levels above.

Ermad avatar Jul 07 '17 22:07 Ermad

Alright, so here are my results for Lv.63-68 recipes using a Lv.63 weaver. I've included the four synth actions from before, and also added basic/standard/advanced touch actions and the Control stat. I'm going to forego generating the goldsmithing data I mentioned earlier unless you absolutely need it, since I want to actually play the game a little bit tonight. XD

Recipe Name Recipe Level iLvl (ffxivcrafting) Craftsmanship Control Basic Touch Standard Touch Adv. Touch C. Synth Basic Synth C. Synth 2 Standard Synth
Ruby Cotton Cap 63 270 539 528 210 262 315 98 109 131 164
Kudzu Thread 64 273 539 528 178 223 267 91 101 121 151
Kudzu Cap of Crafting 65 276 539 528 146 183 220 84 93 112 140
Worsted Yarn 66 279 539 528 115 144 173 76 84 101 126
Serge Turban of Crafting 67 282 539 528 104 131 157 74 82 98 123
Twinthread 68 285 539 528 104 130 157 74 82 98 123
Ruby Cotton Cap 63 270 390 356 150 187 225 71 78 94 118
Kudzu Thread 64 273 390 356 127 159 191 65 72 87 108
Kudzu Cap of Crafting 65 276 390 356 104 131 157 60 67 80 100
Worsted Yarn 66 279 390 356 82 103 123 54 60 72 91
Serge Turban of Crafting 67 282 390 356 74 93 112 53 59 70 88
Twinthread 68 285 390 356 74 93 112 53 59 70 88

crossedxd avatar Jul 08 '17 00:07 crossedxd

Thanks for the data. That confirms for me that progress gets a 2.5% penalty per recipe level up to max 25%, and quality gets a 5% penalty per recipe level up to max of 50%.

I think this change to penalties also applies crafters of all level from the new data that I have seen of lower levels.

Ermad avatar Jul 08 '17 01:07 Ermad

Based on that comment referring to the patch notes, I'd say that's a safe assumption to make.

crossedxd avatar Jul 08 '17 01:07 crossedxd

So as you suggested I collected some data with my 30+ crafters, but I'm still having trouble figuring out the new coefficients for recipes below the crafter level:

crafterClass crafterLevel recipeName recipeLevel levelDifference craftsmanship progress
CRP 31 Steel Spear 30 1 159 37
CRP 31 Walnut Cane 29 2 159 39
CRP 31 Elm Macuahuitl 28 3 159 40
CRP 31 Silver Battle Fork 27 4 159 42
CRP 31 Viper-crested Round Shield 26 5 159 44
CUL 36 Herring Ball 35 1 176 41
CUL 36 Acorn Cookie 34 2 176 43
CUL 36 Tuna Miq'abob 33 3 176 45
CUL 36 Forest Miq'abob 32 4 176 47
CUL 36 Dark Vinegar 31 5 176 48
LTW 37 Boarskin Ringbands 36 1 168 39
LTW 37 Boarskin Survival Boots 35 2 168 41
LTW 37 Boarskin Choker 34 3 168 43
LTW 37 Boarskin Himantes 33 4 168 44
LTW 37 Toadskin Belt 32 5 168 46

~~I'll keep trying, but I think the formula's structure may have changed for recipes below the crafter level where the crafter level is <50. Unfortunately if that's the case, I don't have the capability to figure out a whole new formula.~~

With assistance from stackoverflow I now have some python code which I can use for least squares regression which gives me a formula of the form:

progress = Math.round(a * levelDifference + b * craftsmanship + c)

(I realise that least squares regression is also what that big Excel spreadsheet is doing, but while I tried to understand how to do it in Excel I could not grasp it. With the Python code I don't need to understand much, the hardest part was actually figuring out how to put the data in there)

I successfully found using the stackoverflow code a set of values for a, b and c that fit 1 <= levelDifference <= 5, I'm assuming that as with the original code, different values will be needed going forward to larger values of levelDifference, so I will collect more data.~~

It seems like you guys were already on top of this as I see there are new changes.

danj2k avatar Jul 08 '17 07:07 danj2k

That seems accurate to whats already in the simulator, to within a point. The only thing that has seemed to have changed is the penalty for recipes above your level, which we know and looks consistent across all craft levels.

Ermad avatar Jul 08 '17 14:07 Ermad

~~@Ermad I'm not sure if I tested it incorrectly but when I tried, I could not get the existing code in the simulator to output values that matched the data within the required +/- 0.5 margin of error. With the new formula that I have, I can achieve that.~~

~~EDIT: as I'm adding more data I am finding it difficult to get the error margin down to that level. So yeah I don't know.~~

As to how I've been testing it, I forked the project on github (though the fork has now been deleted as I could not see a way to get it to accept the new changes that @Ermad and @doxxx have put in) then cloned it to my home Linux box. The github.io idea seems better though as that can be safely linked to other people.

danj2k avatar Jul 08 '17 16:07 danj2k

@Ermad is the version of the simulator you have up and running up-to-date with the most recent values? I'm going to be doing some crafting today and figured I could cross-reference my results with what the simulator gives me.

crossedxd avatar Jul 08 '17 17:07 crossedxd

@danj2k I have not updated either the main or the beta site with Ermad's changes yet.

doxxx avatar Jul 08 '17 17:07 doxxx

You can test my branch out at https://ermad.github.io/ffxiv-craft-opt-web/app/

Ermad avatar Jul 08 '17 17:07 Ermad

Awesome! I didn't know you could do that!

doxxx avatar Jul 08 '17 17:07 doxxx

Yeah, I'll be running some tests this afternoon. Currently planning on recording the expected/actual synth values for a bunch of stuff I'm gonna be making for myself. If there's anything else you'd like recorded, let me know asap.

crossedxd avatar Jul 08 '17 17:07 crossedxd

Ermad's changes are also available on the beta site now. I'm leaving this open for further discussion as necessary,

doxxx avatar Jul 08 '17 19:07 doxxx

Here's some more data:

crafterClass crafterLevel recipeName recipeLevel levelDifference craftsmanship progress
BSM 29 Steel Broadsword 28 1 149 35
BSM 29 Steel Nails 27 2 149 36
BSM 29 Iron Raising Hammer 26 3 149 38
BSM 29 Iron Spatha 25 4 149 40
BSM 29 Bas-relief Iron Saw 24 5 149 41
GSM 26 Ice Brand 25 1 142 33
GSM 26 Silver Spectacles 24 2 142 35
GSM 26 Eye of Fire 23 3 142 36
GSM 26 Eye of Ice 22 4 142 38
GSM 26 Brass Choker 21 5 142 39
WVR 32 Velveteen Sash 31 1 141 33
WVR 32 Velveteen Dress Shoes 30 2 141 34
WVR 32 Velveteen Shortgloves 29 3 141 36
WVR 32 Velveteen Half Apron 28 4 141 38
WVR 32 Velveteen Tights 27 5 141 39

danj2k avatar Jul 08 '17 19:07 danj2k

@doxxx much appreciated; I feel like I'm back in business with the tool up and running again, lol.

crossedxd avatar Jul 08 '17 20:07 crossedxd

@Ermad Here are some more values I collected this afternoon. It looks like the solver is pretty spot on, and when it's off it's underestimating rather than overestimating (which is good, better safe than sorry).

The one thing that I noticed that seemed particularly off though was the Dew Thread, which had a much higher delta than the other errors.

Item Name iLvl cLvl cType Craftsmanship Control Action Used States Solver Value Actual Value
Holy Water 52 53 ALC 360 369 Careful Synthesis II   114 114
Dew Thread 23 63 WVR 606 609 Careful Synthesis III   290 305
Rainbow Thread 51 63 WVR 606 609 Careful Synthesis III   290 294
Holy Rainbow Cloth 52 63 WVR 606 609 Careful Synthesis III   290 293
Holy Rainbow Hat 130 63 WVR 606 609 Careful Synthesis II   232 234
Holy Rainbow Hat 130 63 WVR 606 609 Careful Synthesis III   290 293
Wyvernskin Workboots 130 52 LTW 343 334 Carefi; Synthesis II Ingenuity 126 131
Wyvernskin Jerkin 133 52 LTW 343 334 Careful Synthesis II   69 69
Wyvernskin Jerkin 133 52 LTW 343 334 Careful Synthesis II Ingenuity 126 130

crossedxd avatar Jul 08 '17 21:07 crossedxd

It seems the bonus for being over leveled isn't being calculated correctly when you vastly over level a craft isn't being calculated correctly. The current calculation is capped at ~56% at 100 levels above, after hitting 50% at 20 levels above. After doing some brief in game testing, you get a ~75% bonus on a level 1 craft hole being max level (289 levels above).

It seems accurate for up to 20 levels above, then slowly drifts off as you over level it. I'll try to gather more data to improve it.

Ermad avatar Jul 08 '17 22:07 Ermad

@crossedxd that'll be because it's so far below your level - smaller errors that you might not notice around your level are compounded as you go further in each direction.

danj2k avatar Jul 08 '17 22:07 danj2k

I don't have the materials to really test this out in depth, but there's a big issue when it comes to a particular synth I did today.

I'm currently a level 67 Armorer with 1046 craftsmanship and Ingenuity II.

I used a certain rotation I use for making most items right now, and it failed when making the Endless Expanse Shield which is a 60 ** synth. The simulator was saying each of my Careful Synthesis II would gain 397 progress, but the actual result was FAR lower than that at 205. Despite having steps of leeway, such a drastic difference bombed my macro with a failed synthesis.

I wish I could gather more data, but I don't have the resources to do that right now. Just letting everyone know about the drastic difference in that calculation.

EDIT: I wouldn't mind putting in the work to give the data that's needed to impove things, I just need some time to gather the gil and materials, but also direction on what exactly to test to find out where the issue is.

Continuity28 avatar Jul 10 '17 23:07 Continuity28

I just tested a 60** on my 70 Armorer, taking off some gear to get 908 Craftsmanship. I got 352 progrees from a Careful Synth II (both with and without Ingen2). I don't see how you could have gotten 205 in game.

That value is slightly off from what simulator says, 341 and 361 with and without Ingen2. I know it isn't completely accurate when you vastly over level the craft (more then 20 recipe levels), but shouldn't be off me more then like 10% or so. I plan to try to gather a lot of data on that to try to improve it.

Ermad avatar Jul 11 '17 00:07 Ermad

Ah, I found the issue. The recipe "Endless Expanse Shield" has an elemental Aspect: Wind. Apparently when something has an aspect it reduces the progress of all actions by 50% that aren't a Brand of (Aspect Element). This isn't currently modeled at all in the simulator, but I'll look into implementing it. It only effects a very small amount of recipes, only "Vintage" recipes and crafted primal battle items.

Ermad avatar Jul 11 '17 00:07 Ermad

As soon as the craft started and I saw the wind swirling, I actually had that thought, but I forgot all about it.

You're right, it's not that important overall, as so few recipes make use of that, and I just happened to stumble into it. :P

Continuity28 avatar Jul 11 '17 04:07 Continuity28