Text fields

[pmconne]

Is your feature request related to a problem? Please describe. Users have a need to place text that displays a stringified representation of the property of some other object (usually an element). For example, title blocks on sheets often contain tables that are to be filled in with a last modified date, a sheet number, etc.

In MicroStation this is accomplished using TextFields inside of a TextBlock. A TextField describes how to obtain and format the property. The displayed text changes automatically when the value of the property changes.

Describe the solution you'd like A similar solution to MicroStation. TextField would become another kind of Run within a TextBlock.

Describe alternatives you've considered n/a

Additional context Need to identify what kinds of properties and formatting people need us to support.

We probably need the concept of "placeholder fields". For example, a template describing a sheet's title block will have areas where data like the project name, sheet number, etc need to be filled in. Those property values can only be resolved once we place an instance of the title block into a specific sheet, at which point the fields would need to be mapped to the containing iModel's root subject, or the containing sheet model, or whatever.

It's likely that some properties associated with fields will be "ad-hoc" properties stored in jsonProperties, in which case we will lack the type information provided by properties that are stored as first-class BIS properties. That may complicate the formatting logic - the user will probably have to tell us what type (number, date/time, string, distance, area, etc) to expect.

[Josh-Schifter]

How can I help to move this project forward?

[pmconne]

How can I help to move this project forward?

Detail the specific non-technical requirements. Ideally specify what's needed for MVP so we can prioritize. There are some questions in the PR description serving as examples of details needed - formatting? Placeholder fields? etc.

[Josh-Schifter]

We think the first use case for fields will be for title blocks. User requirements for MVP are documented here: https://bentleysystems1.aha.io/features/ITSDP-17

Placeholder Fields: Because of the way we are doing sheet templates, where a sheet template is a sheet, I don't think there's any requirement for placeholder fields. It seems to me that placeholder fields are more aligned with a 'place cell' sort of workflow.

Formatting: Based on the user requirements, it looks to me like the initial formatting requirements would focus mostly on strings (so capitalization) and potentially dates.

[pmconne]

Thanks. We do need to think a little beyond MVP when deciding how to implement this.

Should we assume fields should behave like they do in MicroStation unless otherwise specified? e.g., invalid field displays as "####", fields automatically update as soon as the target property changes, etc? Are there features of fields in MicroStation that we don't want in iTwin.js?

Properties can be stored directly on the element, or its aspects, or a model (separately from the modeled element), or in JSON properties, or in relationships, or in non-ECSql tables like be_Props (@khanaffan did I miss any other possibilities?) Do we need to support all of these different kinds? Do we need to support fields showing computed values not directly tied to a single property?

Implementation questions

Will changes to all kinds of properties trigger a dependency callback? I don't think relationship property changes will.

[Josh-Schifter]

I think working as in MicroStation is a good starting point. I'm certainly not aware of every feature so I can't say what the right answer is up front.

Regarding which specific properties to support, I would suggest starting small since we can add later. In MicroStation, I think we basically stored a property name, an element Id, and probably a display path. Seems to me that the iModel equivalent would be an ecsql query.

[Josh-Schifter]

Has there been any progress on this? If not, is there a plan to begin work on this? If not, how do I escalate this?

[pmconne]

Has there been any progress on this? If not, is there a plan to begin work on this? If not, how do I escalate this?

@aruniverse @calebmshafer

[hl662]

Formatting: Based on the user requirements, it looks to me like the initial formatting requirements would focus mostly on strings (so capitalization) and potentially dates.

It seems like basic formatting APIs available in Javascript is enough here, but long term are you thinking of formatting measurements and the like? That would determine if the text fields themselves should have a KindOfQuantity associated to it, to be used for formatting... Every ECProperty should have a KindOfQuantity property anyways, can that be used?

[pmconne]

@hl662 here is some context on MicroStation's formatting options for fields.

[hl662]

After looking at Microstation's way, it sounds like you'd want a TextField to have it's own formatting based on what a user decides? Independent from any formatting associated to a KindOfQuantity? Consequently, the TextField would have to store it's own FormatProps jsonObject...

[pmconne]

That is correct, the formatting is a property of the field.

[pmconne]

Here are my preliminary thoughts on how to put this together.

BisCore ECSchema changes:

• Add ITextAnnotation, a mixin class that identifies an element that can host text annotations (e.g., TextAnnotation2d/3d, TableAnnotation, DimensionAnnotation, etc). It has no additional properties.
• Add ElementDrivesTextAnnotation, a subclass of the ElementDrivesElement relationship that relates a source Element to a target ITextAnnotation. It has no additional properties.
• Add ITextAnnotation as a base class to TextAnnotation2d and TextAnnotation3d

core-backend changes:

• Add a new interface for ITextAnnotation that is expected to be implemented by all subclasses of the bis:ITextAnnotation mix-in. For now, it only defines a single method, invoked when the source element of an ElementDrivesTextAnnotation relationship is modified or deleted.
• Implement this interface on TextAnnotation2/3d to re-evaluate any field(s) associated with the source element.
• Add a class representing bis:ElementDrivesTextAnnotation and implement onRootChanged and onDeletedDependency to cast the target element to ITextAnnotation and invoke the re-evaluate fields method.
• When inserting or updating a TextAnnotation2/3d, create and/or delete the ElementDrivesTextAnnotation relationships required by the fields it contains (one per source element).

TextBlock changes:

• Add FieldRun as a new type of Run. It stores the element Id of the source element that hosts the property from which the run's display string will be derived, an accessor describing the property of interest, and a description of how to format the property value for display.
• Add TextBlock.reevaluateFields(sourceElement: Element): number. For each FieldRun that refers to the specified element, it recomputes the display string; it returns the number of display strings that differ from the stored display string. The caller can then perform layout if any fields actually changed.

NOTE: ElementDrivesElement prohibits circular dependencies. Therefore, unlike in MicroStation, an annotation cannot contain a field that displays a property of that same annotation. Though I suppose an annotation element could manually re-evaluate its fields during an update to check if a self-targeting field had changed.

Questions:

• MicroStation uses "####" to indicate a field that could not be evaluated (property does not exist, source element deleted, etc). Stick with that?
• MicroStation rendered fields with a grey background. Do we want that, or some other way of distinguishing them from ordinary text? MicroStation provided a run-time option to turn this background off; that would be trickier with iTwin.js' tiled graphics.
• The source property must ultimately resolve to a primitive type, but it may be nested inside structs and/or arrays. We need some kind of "property path" concept to express this. People might want to specify negative array indices (e.g., items[-1].unitCost for the last item in an array).
• The source property must come from the source element, but it may be a property of an aspect of that element. Because multi-aspects exist, we can't simply specify an aspect class name. How to permit the user to specify which aspect they want? Some kind of mini-query that lets them select among aspects by their property values? A realistic use case I can foresee would be people who want to display the DGN filename or element Id, which comes from ExternalSourceAspect (a multi-aspect). We can probably punt on aspects to start with and add them later without breaking the API/persistance.
• We probably need to support properties of JSON properties. How will we determine the property type so that we know how to format it? Perhaps the user just has to tell us.
• How to implement placeholder fields? (We'll need them for things like title blocks).
• How to describe formatting? The options will differ based on property type. MicroStation ships with a MstnPropertyFormatter ECSchema it uses for this purpose; we should start there.

[ColinKerr]

@grigasp The text fields discussed here reference properties in related elements. It would be interesting to see how your property paths compare to the property paths proposed for text fields. I found this documentation on relationship path specification: https://www.itwinjs.org/presentation/relationshippathspecification/. Is this the appropriate thing for us to compare to?

@pmconne can you point to the current WIP format for your property paths?

[pmconne]

@pmconne can you point to the current WIP format for your property paths?

There isn't one. I need to be able to encode things like:

• topLevelProperty
• struct.property
• array[0]
• structs[0].property
• jsonProperty.objects[1].property And combinations thereof.

It might be simplest to encode it so it can be inserted into an ECSql SELECT statement. ECSql supports JSON queries of arbitrary complexity. I'm not sure if ECSql supports things like SELECT structs[0].array[1].property (@khanaffan?)

[pmconne]

We should avoid storing element Ids in the JSON representation of the fields if feasible, because element Ids may change during transformations and the transformer may not know how to find and replace them. They are also redundant given they will be encoded into the ECRelationship. Can we instead have the field store some stable reference to the ECRelationship that points to the target element? Perhaps add a string property to the ECRelationship class and give it a unique value among all the instances of that relationship pointing to the target element, and store that unique value in the field.

[Alfonso-Martello]

I assume that we should avoid storing element IDs in the JSON representation for text styles as well?

This PR is introducing having the JSON representation of the text refer to AnnotationTextStyles by ID. To avoid this, would we need to introduce a new ECRelationship like for fields? I recall during a past BWG meeting (titled "Text Annotation topics" on 2025-02-07) that we discussed relationships like this existing in the past, but we removed them due to poor performance? Specifically, I recall GeometryParts being mentioned

[pmconne]

I assume that we should avoid storing element IDs in the JSON representation for text styles as well?

I don't think a relationship per text style used in annotation is feasible/desirable. You have to store the Id somewhere. Until someone either comes up with a way to automatically discover+remap element references inside JSON while handlers are not loaded, or a way to ensure required handlers are always available, I think you should do the obvious thing and store them in JSON as proposed in your PR.

[FriederKirn]

FYI - here some context for "data/text fields" with the use case in a title block https://bentleysystems1.aha.io/features/ITSDP-17

[pmconne]

MstnPropertyFormatter.01.00.ecschema.xml.txt is the schema MicroStation uses for specifying field formatting options. The appropriate formatter is selected based on the property's primitive+extended types and custom attributes like StandardValues which define enums (perhaps superseded by ECEnum now).

[pmconne]

For formatting, I think quantity formatting and kinds-of-quantity mostly cover what we need for formatting fields. A few additional requirements for fields:

• Formatting of date/time values. MicroStation (pre-CONNECT) stored C# format strings. MicroStation CONNECT stores some other format string usable in pure C++ (details unclear, though I probably wrote it...need to look at the implementation).
• Specify optional prefix and/or suffix, e.g. "Planet Earth is {field value} away". You would think you could just put the prefix and suffix as separate runs from the field but I think they are not displayed if field value can't be resolved.
• Some people want to specify the casing of strings (unchanged, all upper, all lower, capitalize first word, capitalize each word).
• Boolean values localized as true/false, yes/no, on/off, enabled/disabled.
• Localized enum display values.

For fields pointing to JSON properties the user may want to explicitly specify the KoQ since there's no ECProperty from which to obtain it.

The big issues are:

1. Quantity formatting appears to be an entirely front-end-only concept, and we need to format these values on the back-end.
2. Quantity formatting and units APIs appear to be mostly async and we need to format synchronously in the middle of transaction processing.
3. Localization of booleans and enums. We don't know the locale on the backend, and we can't display the fields in different locales for different users. Maybe the UI for creating a field lets the user choose the localized set of display strings to use and we persist those in the field.

@ColinKerr @hl662 are you the right people to talk to about quantity formatting and units? Is there any inherent obstacle to doing quantity formatting on the backend? Would a single package usable in both frontend and backend contexts be feasible?

[hl662]

@rschili can also chime in too,

Quantity formatting appears to be an entirely front-end-only concept, and we need to format these values on the back-end.

You can format those values on the backend, the APIs and interfaces are available under @itwin/core-quantity. As of 5.0 we have added a SchemaFormatsProvider (in @itwin/ecschema-metadata) which enables you to find the presentation formats of a KindOfQuantity to use in formatting. See the Quantity learning documentation there are code examples of formatting using that, without need for any core-frontend APIs. The example manually enters in a persistence unit, but if the backend knows the KindOfQuantity you could also query the schemaItem and get the persistenceUnit property of that item programatically.

Quantity formatting and units APIs appear to be mostly async and we need to format synchronously in the middle of transaction processing.

For this, the workflow of formatting can be split into two sections: Preparing the formatting spec (the configuration of the formatting) and applying the formatting spec to a value. The former is done asynchronously, while the latter is synchronous (exact method here). With the former, can this process be done ahead of actual formatting, or can it be done using .then.catch() workflows?

For point 3, what do you mean by localized enum display values? Or localization in general?

[pmconne]

With the former, can this process be done ahead of actual formatting, or can it be done using .then.catch() workflows?

Can you link to what you mean by "the former" (the async part)?

For point 3, what do you mean by localized enum display values? Or localization in general?

Example:

    <ECEnumeration typeName="SectionType" backingTypeName="int" isStrict="false">
        <ECEnumerator value="3" name="Section" displayLabel="Section"/>
        <ECEnumerator value="4" name="Detail" displayLabel="Detail"/>
        <ECEnumerator value="5" name="Elevation" displayLabel="Elevation"/>
        <ECEnumerator value="6" name="Plan" displayLabel="Plan"/>
    </ECEnumeration>

The display labels are to be localized. If I point a field at a property of type "SectionType", I expect the field to display the value in some specified language.

[hl662]

Can you link to what you mean by "the former" (the async part)?

From one of the code examples in the learning doc:

const formatsProvider = new SchemaFormatsProvider(schemaContext, "metric");
const unitsProvider = new SchemaUnitProvider(schemaContext);
const persistenceUnit = await unitsProvider.findUnitByName("Units.M"); // or unitsProvider.findUnit("m");

const formatProps = await formatsProvider.getFormat("AecUnits.LENGTH");
const format = await Format.createFromJSON("testFormat", unitsProvider, formatProps!);
const formatSpec = await FormatterSpec.create("TestSpec", format, unitsProvider, persistenceUnit);
// Everything above this is the async part, of generating a formatSpec

// Below here is the synchronous part, actual applying the formatting
const result = formatSpec.applyFormatting(50); // The persistence unit is meters, so this input value is 50 m.

As for localizing the display labels I'm not aware of how to do that either, I ran into the same problem with the display labels for KindOfQuantity, and possibly having to localize that... @ColinKerr ?

[pmconne]

In that case, no. The user calls iModelDb.saveChanges, native TxnManager detects that an element has changed, invokes our JS callback to update the fields that refer to that element's properties (formatting happens here), then finishes processing the transaction. We can't await inside our callback.

Are all those functions necessarily async or could synchronous versions exist?

To be clear, I expect the field will encode all of the formatting info in its JSON representation. We will obtain the KoQ from the ECProperty, or specify it explicitly for JSON properties. We are not interested in applying run-time/per-user formatting preferences to fields.

[hl662]

It's necessary - to create a formatter spec, we do unit lookups and get the unit conversions between units ahead of time. With the UnitProvider being a SchemaUnitProvider, that involves querying the imodel db then. Same with the SchemaFormatsProvider, it queries the iModel for the KoQ's format properties.

Can't the JS callback update the fields and in a .then() continue processing the transaction

[pmconne]

With the UnitProvider being a SchemaUnitProvider, that involves querying the imodel db then.

Which can be done synchronously, our APIs just discourage it. I am synchronously querying the db for the property value and metadata.

Can't the JS callback update the fields and in a .then() continue processing the transaction

No, the database is in an inconsistent state until all transaction processing completes. You can't defer transaction processing and allow some other code to access the database in the interim.

[hl662]

Which can be done synchronously, our APIs just discourage it. I am synchronously querying the db for the property value and metadata.

In that case... maybe...? We'd have to add new synchronous interfaces for the FormatsProvider and UnitsProvider that defines the same methods (just sync versions), and on the ec metadata side, have a different (Synchronous)SchemaUnitProvider. Within the unit conversion code of the current SchemaUnitProvider there's something about a resolveUnit method that I'm not familiar with and not sure if that can be converted to sync. Which I will let @ColinKerr answer

[pmconne]

We'd have to add new synchronous interfaces for the FormatsProvider and UnitsProvider

Probably leave the base interfaces alone, on the back-end I'm going to have to instantiate a specific implementation anyway (or maybe just call a couple new standalone synchronous functions), there's no equivalent to IModelApp.formatsProvider AFAICT.

[hl662]

On the core-quantity side, I know the code example above uses await FormatterSpec.create("TestSpec", format, unitsProvider, persistenceUnit); It's a static helper method that generates a list of unit conversion objects (just that the method to generate them is via the asynchronous unitsProvider.getUnitConversions).

If you have a specific synchronous implementation of the unitsProvider that can get the unitConversions, you'll be good to instantiate a FormatterSpec without using the helper method (see ctor).

So yeah, the single package needed is just @itwin/core-quantity