recipes
recipes copied to clipboard
TandoorRecipes
Food Unit Rework Proposal
Is your feature request related to a problem? Please describe.
Units are currently just integers with text labels. While this is simple and very easy to use, it makes it hard for the system to reason about amounts.
Ideally the following things should be possible.
- Opportunistic Conversions between units for clarity (ie, 0.0625 pound to 1 ounce)
- Conversion from informal units to concrete units (ie, 1 medium yellow onion to 100 grams of onion)
- Allowing for space wide "custom conversions" (A basket of strawberries on your farm is 1000 grams)
Describe the solution you'd like
- Use the pint as the ingestion and conversion library for concrete units where possible.
- Use the gram as the base unit for storage when possible.
- Keep a unit model in the database, with a separate FoodUnitRelation model to contain food specific cross type conversions.
- Categorize Units by type (Volumetric vs Weight vs Informal) via a classification field on unit
- Implement as much of the conversion and equivalency logic in a set of "fat" models and expose that via a list of "alternative" units to each unit returned.
Describe alternatives you've considered
- just using pint directly, discarded because of the lack of informal and colloquial units.
- Considered requiring all metric weight based recipe entries, but that would not allow for easy recipe imports.
- hard coded conversion lookup table, but that lacks flexibility and once again limits imports.
Additional context
Planned Nice to haves
- Automatically simplify fractional inputs (ie, 1/2 cup becomes .5 cup automatically at least internally)
- Per space and per user unit presentation preferences separate from internal storage of underlying unit information(UserA uses weight in grams, UserB prefers American common volumetric units)
- Automatic recipe unit normalization
- Add a one click button to pull in Informal amounts and per food conversions from the https://fdc.nal.usda.gov which contains information in the format of
{
"foodClass":"FinalFood",
"description":"Tomatoes, grape, raw",
"foodPortions": [
{
"id": 118808,
"measureUnit": {
"id": 1082,
"name": "tomatoes",
"abbreviation": "tomatoes"
},
"modifier": "",
"gramWeight": 49.7,
"sequenceNumber": 1,
"minYearAcquired": 2016
},
{
"id": 118809,
"measureUnit": {
"id": 1000,
"name": "cup",
"abbreviation": "cup"
},
"modifier": "",
"gramWeight": 152.0,
"sequenceNumber": 2,
"minYearAcquired": 2016
}
]
}
Awesome proposal! Definitely looks to me like a well-planned idea.
You propose the gram as general base unit, correct? Does it make sense, to use ml for liquids in the same fashion and then translate on the fly?
The fractional input is about formatting when editing a recipe? As far as I know, the conversion already happens automatically when importing.
I would propose to think about the importing as a separate issue, as this could be done by having different importers for different sources. E.g. importing from FDDB or from fdc.nas.usda.gov Should be easy to do if the unit framework is implemented in a sensible way.
I agree the recipe import enhancements might end up in its own ticket, that is why I have it in nice to have. If i can roll it into my pr for this i will, but if it gets complicated I would spin off a separate ticket and pr. I might have run into issues with fractions that have been touched by ms word style "smart" fractions where 1/2 gets replaced by unicode U+00BD and made a false assumption.
re: gram vs ml, Not all ingredients have a meaningful volumetric measurement that I can calculate, but I think just about everything can be related to weight, even liquids of various densities. So for storage sake, ideally, i would want to reduce everything to grams on the backend.
Sounds very very good. I am currently on my phone but will leave a few comments later.
ok so i am on a PC now, some comments in random order
- informal to formal units, very important, can be attatched to each food as you said as a "conversion table"
Cleaner input of recipes which list multiple amounts for a ingredient in different systems (ie, "2–3 cups (375g) frozen strawberries" gets stored as "375g of strawberries" )
not sure how you want to do that. The discussion about having "from-to" ingredients has come up quite often but i personally have not found a "good" solution. I dont really like to add another field as the editor is already way to complex so until I or someone else proposes something really good i prefer "forcing" the user to just decide on an amount
Allowing for space wide "custom conversions" (A basket of strawberries on your farm is 1000 grams) This is nice, probably not for food realated stuff but if you want to have custom units, like a special object you have in your kitchen that can carry for example 35g or whatever. Not really a highly important thing but nice to have.
Use the gram as the base unit for storage when possible. i dont think i would make that change, i would always save whatever the user saves but i would allow for some kind of unit presedence. So you could give "g" rank 1 and "kg" rank 2 so whenever pint or a custom conversion is able to convert into g or kg it will and if not it wont. is this understandable what i mean ?
Categorize Units by type (Volumetric vs Weight vs Informal) via a classification field on unit nice but not sure how much this helps, maybe for genereic presendence (i.e. weight > volume > informal ?)
Implement as much of the conversion and equivalency logic in a set of "fat" models and expose that via a list of "alternative" units to each unit returned.
how exactly do you mean this ? i think i would implement the conversion in such a way that the ingredient is return with the default unit unchanged to the way it was til now but also include a "alternative_units" (or something similar) array with the amount and unit that can be converted. this would not require a model as i would calculate it on the fly. on thing to consider here is that pint can have many conversions for one unit so maybe we need to limit it to the highest ranks.
Automatically simplify fractional inputs (ie, 1/2 cup becomes .5 cup automatically at least internally) already the case, fractions become decimals
Per space and per user unit presentation preferences separate from internal storage of underlying unit information(UserA uses weight in grams, UserB prefers American common volumetric units)
hmm probably per user because per space is already in the food/unit database. But that is also very complicated, maybe one could use the classifications you proposed above, maybe add another "imperial" / "metric" instead of Volumetric and Weight and let the user decide the presedence between the different types (that could maybe also save the effort of adding individual unit ranks)
sorry for the very long comment but i have so many toughts about this, if you have any questions let me know
Long comment is completely reasonable. I am going to respond to your questions in order via numbers if thats ok, to avoid flooding you with quotes of quotes.
- I should change that line to 'easing importation'. i don't want to make changes to the editor to handle that case. I think it should just pick which ever is first. I chose a bad example because that is what I happened to run into, and the metric measure got shoved into the food field.
- I wanted to do it via a many to many conversion table (FoodUnitRelation) with data pulled in from that us gov json blob. Standard conversions that are not cross type I would handle through the pint lib. For example, a pound of cherry tomatoes are the same in grams as a pound of strawberries, but a cup of each weigh different amounts, so you would have to go through a FoodUnitRelation for each to convert to grams.
- I agree that we could add the option for metric volumetric and imperial weights, but those two would probably be the least commonly used preferences. No reason not to add it if people want it.
Moved the bit described in question 1 into https://github.com/TandoorRecipes/recipes/issues/1959
re: question 2, upon further thought I can be convinced to leave original units in the recipe and convert to grams at run time, downside for that comes later when you want to do things like search for foods that fit your macro's for the next week or use up the rest of a ingredient, But the downside is just speed, as you could still search for each possible conversion.
Also in so far as deciding unit priorities, it might make sense to have a table of preferred units for each common recipe unit class (Imperial Volume, Metric Weight, etc) as you mentioned but then we could just see what units show a integer result. 465g is better than .456kg, A single liquid ounce rather than .0625 of a pint, at least when we are scaling a recipe up and down.
What i meant as far as models is that I would stick functions onto the unit and food unit models to handle the logic, when returned to js/the api it would just be a list of unit's and amounts the client side could select to present. I would avoid implementing any unit handling logic in js.
i perfectly agree with basically everything you said. Take a crack if you want and if you need help at any point let me know, also if you are uncertain and just want to discuss a solution feel free to post here or chat on discord or somethign.
I wanted to do it via a many to many conversion table (FoodUnitRelation) with data pulled in from that us gov json blob. Standard conversions that are not cross type I would handle through the pint lib.
For example, a pound of cherry tomatoes are the same in grams as a pound of strawberries, but a cup of each weigh different amounts, so you would have to go through a FoodUnitRelation for each to convert to grams.
I would strongly recommend against using this approach (along with converting to grams at runtime) for a couple reasons, but the top two are:
- Units are not uniform across the globe; Australian cup and US/UK/Australia/Finnish/Swedish tablespoons are not the same thing.
- Food is a free form text field across multiple languages. Adding yet another metafield to maintain and forcing all users into maintaining reference data in English is not a good user experience.
I would suggest having a canonical unit that is stored in the database for each ingredient - grams and litres (these are, after all, literal international standard units).
When a user enters an ingredient, they can enter it using whatever unit they wish. When the recipe is saved/updated, all ingredients are converted at runtime into g/l units for the database. Each entry in the recipe for each ingredient could then have a default 'display unit' which is the unit the user entered it as. When the recipe is displayed, the unit is converted at runtime from g/l back to whatever the display unit should be.
This would also allow changing display units on the fly, and allowing a user to select a default 'units profile' (Metric, US, AU, whatever) for all recipes that could override the default set by whoever entered the recipe in the first place.
You could also use that to create unit profiles for different types of recipe - for example, I like SI units for most things, but prefer cups for baking as it's far easier measuring out flour that way.
Could the software only convert those units that do conform? Then those that are not mass or weight or volume would stay the same and those that are would change?
Could the software only convert those units that do conform? Then those that are not mass or weight or volume would stay the same and those that are would change?
The software would need to know - in advance, what measurements, in every language, are masses and volumes. It sounds totally unworkable to me.
Only if you are planning on doing it yourself. Given gathering information like this doesn't involve any kind of coding, you could easily ask the community for help.
Failing that, again, you don't have to do the work. Force a default set of units (SI) and let the user add their own, with conversions, as necessary, both generally and per-ingredient.
I'm confused what you mean by 'not all units are mass or weight' - could you elaborate?
Failing that, again, you don't have to do the work. Force a default set of units (SI) and let the user add their own, with conversions, as necessary, both generally and per-ingredient.
making tandoor more difficult or less pleasant to use is definitely not on the roadmap.
I'm confused what you mean by 'not all units are mass or weight' - could you elaborate? Whole, each, half, piece, can, length, wedge, slice off the top of my head from my own set of units.
I agree about making it more difficult. So crowd source a number of regional profiles and offer them as an option, along with the ability to create and share their own. That covers all bases.
I hadn't thought about those kinds of 'units' because they seem trivial to solve - just don't give them a unit type at all in the back end. If you are divorcing what the user sees from what the database stores with a translation layer in the middle (with translations per ingredient), then anything without a unit type stored in the database strictly uses whatever is set for the ingredient.
US Profile:
Unit Type | Profile Unit | Conversion
----------+---------------+-----------
m | oz | 0.035
v | US gal. | 0.263
v | cup | 4.228
m | lb | 0.002
Recipe:
Ingredient | Qty | Type | Unit | Interface
------------+-------+-------+---------+------------------
flour | 100 | m | US | 3.53 oz flour
milk | 4 | v | US | 1.057 gal milk
onion | 1 | | | 1 onion
lemon | .5 | | | 1/2 lemon
chocolate | 6 | | piece | 6 pieces chocolate
beans | 1 | | can | 1 can beans
ginseng | 1 | | inch | 1 inch gingseng
orange | 2 | | wedge | 2 wedges orange
pepperoni | 10 | | slice | 10 slices pepperoni
I agree about making it more difficult. So crowd source a number of regional profiles and offer them as an option, along with the ability to create and share their own. That covers all bases.
That creates a bad user experience. Most users are casual - they find a recipe on a website that they want to import into Tandoor with minimal fuss. Forcing them to do a bunch of data management (which Tandoor, in general, already does too much of) makes the entire application less pleasurable to use.
I hadn't thought about those kinds of 'units' because they seem trivial to solve - just don't give them a unit type at all in the back end.
Units are far more difficult than you are considering. Tablespoons (and maybe teaspoons?) have at least 4 different volumes depending on region. Ounces can be mass or volume depending on context. Changing the actual measurements without user intention is going to cause all sorts of problems, which destroys the user experience.
Additionally, you can't just 'ignore' units that aren't on the backend - you will end up with inconsistent user experiences based on the availability of translations, and regionality, which again, destroys user experience.
That creates a bad user experience. Most users are casual - they find a recipe on a website that they want to import into Tandoor with minimal fuss. Forcing them to do a bunch of data management (which Tandoor, in general, already does too much of) makes the entire application less pleasurable to use.
I fail to see where you think the user has to do more data management? It could be something as simple as a drop down box in settings to select one of the crowd sources profiles, in the same way they can choose light or dark mode...
Tablespoons (and maybe teaspoons?) have at least 4 different volumes depending on region.
That's literally what I'm talking about by setting up profiles...
Ounces can be mass or volume depending on context.
That's why I put in a unit type...
Changing the actual measurements without user intention is going to cause all sorts of problems, which destroys the user experience.
I didn't once talk about changing actual measurements. I'm talking about a standard behind-the-scenes measurement, and a conversion between that and whatever the user has chosen.
Additionally, you can't just 'ignore' units that aren't on the backend - you will end up with inconsistent user experiences based on the availability of translations, and regionality, which again, destroys user experience.
I'm not ignoring units, I'm ignoring unit types. If it's not mass or volume, just do whatever the recipe says.
It really seems like you haven't understood what I've suggested, are there particular points I can elaborate on for you?
Tablespoons (and maybe teaspoons?) have at least 4 different volumes depending on region.
That's literally what I'm talking about by setting up profiles...
And what happens if a user doesn't know that they need to setup a profile? Or they just have the default?
Ounces can be mass or volume depending on context.
That's why I put in a unit type...
The importer doesn't know what the unit type is.
It really seems like you haven't understood what I've suggested, are there particular points I can elaborate on for you?
To be honest, I don't think you really understand the complexity of what you are proposing. You need to outline how data migrations work, both existing, new, as new profiles are created, as profiles are changed. How to handle multi-user scenarios within the same space, what happens when a user changes "regional" profiles. How to handle data migrations if a profile is changed/corrected.
How to handle multi-user scenarios within the same space
See this is why I wonder if you don't know what I'm talking about. The user profile is what determines how the data is displayed. Different users can display different units. If you put a recipe up using US, that gets translated into generic in the back end, and then I can select AU units, so the generic is converted for display.
what happens when a user changes "regional" profiles
Again, you don't get it. You could be looking at a recipe, and change the profile on the fly and have the units convert to the new profile. I mean hell, half the existing recipe websites can already do that from US to metric!
To be honest, I don't think you really understand the complexity of what you are proposing. You need to outline how data migrations work, both existing, new, as new profiles are created, as profiles are changed.
I think you are vastly complicated matters in your own head, which is why I was asking if you need some help working through this. The profile is nothing more than how the data is displayed. The back end data does not alter. There is no 'data migration' between profiles. It would be nothing more complicated then selecting a visual theme - it's looking up a unit conversion table.
If you are talking about actual database data migration from what is currently stored to whatever the final database schema will be, well, you don't even think about that until the schema has been finalised.
I think you are vastly complicated matters in your own head, which is why I was asking if you need some help working through this. The profile is nothing more than how the data is displayed. The back end data does not alter. There is no 'data migration' between profiles. It would be nothing more complicated then selecting a visual theme - it's looking up a unit conversion table.
If you are talking about actual database data migration from what is currently stored to whatever the final database schema will be, well, you don't even think about that until the schema has been finalised.
This is why I'm pretty sure you aren't aware of how complex what you are suggesting actually is. You are assuming that the data in the database is correct. I'm suggesting that you will never get the data in the DB to be accurate based on the scenarios I've outlined. You are making some assumptions that don't hold up in the real world.
Perhaps I'm wrong about that complexity - but to change my mind you are going to need to walk through the process from data import to data storage. And those scenarios need to consider every state of the 'profiles' and new/updated/existing recipes as updated/changed by each type of user.
Thanks for all the input. I recently played around integrating with a third party cooking machine and they had hardcoded units only (like 10 or so). This of course limits the user freedom for units but on the other hand makes things like conversion, .. much much easier.
As smilerz mentioned, we need to keep the user freedom to add arbitrary units if the user likes to since many websites contain lots of different units. On the other hand I am slowly thinking that units should not be newly created without the user noticing.
Besides the proposed data structure above which to me still seems like a good solution we should maybe add some point add some kind of UI indication if a unit is newly created and allow to easily find a "standard" or "existent" unit that might be better .
Besides the proposed data structure above which to me still seems like a good solution we should maybe add some point add some kind of UI indication if a unit is newly created and allow to easily find a "standard" or "existent" unit that might be better .
I like the friction on transparently creating new units.
The “standard” units should be a secondary attribute on existing units. You don’t have to worry about existing unit and it allows for the creation of custom units (medium onion to 100g as above). You also avoid all of the complexity of migrating existing data and avoiding making wrong assumptions on how to treat existing units.
it limits all of the complexity on write
@smilerz
You are assuming that the data in the database is correct.
I'm talking about an entirely different schema, so I am not considering any existing data at all. To do so makes one get bogged down in details that may be irrelevant or trivial to solve. Like I said before, design the new schema to do what is required, and only then consider data migration afterward. To do anything else hampers productivity when creating the new schema - as you have amply demonstrated.
Perhaps I'm wrong about that complexity - but to change my mind you are going to need to walk through the process from data import to data storage.
I have, twice. That's why I think you are blinding yourself with your own assumptions about existing data and not engaging with me in an informed way about the schema itself from a fresh perspective.
@vabene1111
I recently played around integrating with a third party cooking machine and they had hardcoded units only (like 10 or so). This of course limits the user freedom for units but on the other hand makes things like conversion, .. much much easier.
Now imagine there was someone sitting between you and the machine doing unit conversion for you on the fly. That's all I'm suggesting here. You tell them '15 of my units', the person converts that into '26 machine units'. Then the machine says 'add 4 more machine units', and the person reads that and says to you 'you need to add 2 more of your units'.
In terms of this software, it could be nothing more than a json string of conversion units run by javascript on the webpage. The json string constructed from a user-selected (or created) profile that is stored in the database. You could change profile on the fly with a drop down box - fetch the new json string, apply it to the javascript rendering the ingredients list.
As smilerz mentioned, we need to keep the user freedom to add arbitrary units if the user likes to since many websites contain lots of different units.
I've addressed that. Keep them as-is (don't assign them a unit type), or, if a user wants to deliberately convert units (say, 1 large onion in a domestic recipe to 100g of onion for use in a catering environment where they have access to pre-prepared ingredients) then again, that's not that difficult to do. It gets stored in the backend as 100 mass units (i.e. 100 grams), and the profile unit is 100 mass units to 1 large, resulting in it being displayed as 1 large onion or 100g of onion depending on the unit profile chosen. Given what a 'large' onion is is entirely arbitrary and random anyway, any conversion the user explicitly adds will be acceptable - they are the one adding it, after all!
@smilerz
The “standard” units should be a secondary attribute on existing units.
This is an absolutely terrible idea, and goes entirely against everything in schema design. You do not store calculated values. You store one value, and the conversion factor. Otherwise you're going to have huge problems down the line. What if a user decides to update the recipe from 1 medium onion to 1 large onion, and doesn't change the 'secondary' unit value, as you call it?
it limits all of the complexity on write
I think this might have been a note for expansion, but it actually introduces complexity on write because it's writing data that is unnecessary.
Your users aren't dumb. Fixating on this 'don't make it harder' is both disrespectful to users, and preventing good solutions to problems, especially when solutions incorporate defaults, work-arounds, and crowd-sourced help.
@smilerz
You are assuming that the data in the database is correct.
I'm talking about an entirely different schema, so I am not considering any existing data at all. To do so makes one get bogged down in details that may be irrelevant or trivial to solve. Like I said before, design the new schema to do what is required, and only then consider data migration afterward. To do anything else hampers productivity when creating the new schema - as you have amply demonstrated.
Perhaps I'm wrong about that complexity - but to change my mind you are going to need to walk through the process from data import to data storage.
I have, twice. That's why I think you are blinding yourself with your own assumptions about existing data and not engaging with me in an informed way about the schema itself from a fresh perspective.
The schema is irrelevant. You are proposing rewriting data on write with no discussion at all about how to accomplish that. I’m telling you it is unworkable hand waving it away and refusing to discuss it is t going to change my mind.
Ok thanks for all the input, I think the discussion made the ideas clear.
I have a pretty good idea on how I would implement unit conversion and the first few comments I think reflect this pretty well. Everyone is welcome to give it a try based on that or alternatively wait for me to do it.
The schema is irrelevant. You are proposing rewriting data on write with no discussion at all about how to accomplish that. I’m telling you it is unworkable hand waving it away and refusing to discuss it is t going to change my mind.
sigh
For the third(!) time...
On Input: User has selected a conversion profile. User puts quantity into an input box, and selects user units. On submission, user units are converted, using the rules of the conversion profile, into standard units. Standard units are stored in the database.
On Output: User has selected a conversion profile. User selects a recipe to view. Standard units are fetched from the database, and converted, using the rules of the conversion profile, into user units. User units are displayed on the recipe.
It really isn't hard to understand, I'm not sure where you find it difficult?
The schema is irrelevant. You are proposing rewriting data on write with no discussion at all about how to accomplish that. I’m telling you it is unworkable hand waving it away and refusing to discuss it is t going to change my mind.
sigh
For the third(!) time...
On Input: User has selected a conversion profile. User puts quantity into an input box, and selects user units. On submission, user units are converted, using the rules of the conversion profile, into standard units. Standard units are stored in the database.
On Output: User has selected a conversion profile. User selects a recipe to view. Standard units are fetched from the database, and converted, using the rules of the conversion profile, into user units. User units are displayed on the recipe.
It really isn't hard to understand, I'm not sure where you find it difficult?
It's also not hard to think about all of the ways that it can go wrong. You are making lots of assumptions to make this work reliably - it might help you think through it if you document all of those assumptions and imagine how they may or may not hold true.