Should Intl APIs match Temporal interpretation of `calendar` and `timeZone` options?

[justingrant]

(Following up from discussion of https://github.com/tc39/proposal-temporal/issues/2005 at the 2023-01-05 Temporal Champions meeting, at @sffc's request I'm filing an issue here in 402)

Temporal allows callers to provide calendar and/or timeZone properties in any of the following ways:

1. a string ID, e.g. 'gregory' or 'America/Los_Angeles'
2. an object which can be coerced to a string ID, which could be a Temporal.TimeZone or Temporal.Calendar object or any other object which can be coerced to a string ID
3. an ISO string that contains a calendar and/or time zone annotation, e.g. 2022-01-05T00:00Z[America/Los_Angeles][u-ca=gregory]
4. as a Temporal object that contains a calendar (PlainDateTime, PlainDate, PlainYearMonth, PlainMonthDay, or ZonedDateTime) or a time zone (ZonedDateTime)/

The ToTemporalTimeZone and ToTemporalCalendar AOs are used to handle all 4 cases above.

Currently, Intl (and hence toLocaleString methods of Temporal objects) supports (1) and (2) only. It does not support ISO strings nor property bag option for calendar nor timeZone.

This issue is a request to make toLocaleString methods of Temporal objects support cases (3) and (4) above, so that they'll be consistent with consistent with all other Temporal APIs. I assume it'd make sense for Intl.DateTimeFormat to adopt the same behavior too.

Currently, Intl.DateTimeFormat uses the following spec text to coerce calendar and timeZone to strings:

6. Let _calendar_ be ? GetOption(_options_, *"calendar"*, *"string"*, *undefined*, *undefined*).
...
29. Let _timeZone_ be ? Get(_options_, *"timeZone"*).
30. If _timeZone_ is *undefined*, then
  a. Set _timeZone_ to DefaultTimeZone().
31. Else,
  a. Set _timeZone_ to ? ToString(_timeZone_).
  b. If <del>the result of IsValidTimeZoneName(_timeZone_)</del><ins>IsAvailableTimeZoneName(_timeZone_)</ins> is *false*, then
    i. Throw a *RangeError* exception.
  c. Set _timeZone_ to ! CanonicalizeTimeZoneName(_timeZone_).

One possible solution could be to change the spec text above to first call ToTemporalTimeZone and ToTemporalCalendar on the option values before coercing them to strings.

Note that there's no need to change the internal slots of Intl.DateTimeFormat of the string-typed internal slots that correspond to the constructor's calendar nor timeZone options. Rather, this change would simply change how those options are translated into string slots.

AFAICT, this change would be web-compatible, because cases (3) and (4) above currently cause exceptions today. See https://github.com/tc39/proposal-temporal/issues/2005 for more details and code samples.

[FrankYFTang]

@justingrant Could you explain what are the use cases for Temporal to support case 3 and 4?

@zbraniecki

[FrankYFTang]

I assume if TG2 agree with this, the plan is for Justin to propose a PR to change the section "15.4.1 InitializeDateTimeFormat " in the Temporal proposal , it that correct?

https://tc39.es/proposal-temporal/#sec-temporal-initializedatetimeformat

[justingrant]

Could you explain what are the use cases for Temporal to support case 3 and 4?

There are two general principles used throughout Temporal that apply here:

• a) When a Temporal method accepts a particular Temporal type as a parameter, objects of a different Temporal type can also be accepted, as long as the provided type includes a superset of properties of the intended type. For example, you can pass a Temporal.ZonedDateTime to a method that accepts a Temporal.PlainDate.
• b) Anywhere a Temporal object is expected by a method, the caller can also pass a string or property bag that can be coerced (using the internal equivalent of from) to a Temporal object of the same type. So methods that accept a Temporal.PlainDate parameter can also be passed 2020-01-01 or { year: 2020, month: 1, day: 1 }.

The reasons for both of these principles were to make Temporal calls more ergonomic:

• Date/time data is often more ergonomically described as a string or a property bag instead of a constructed object. For example, time.since('12:00') or time.since({hour: 12}) is more ergonomic than time.since(Temporal.Time.from('12:00') or time.since(Temporal.Time.from({ hour: 12 }).
• Passing superset types is more ergonomic than requiring downcasting every time. For example:

const xmas = Temporal.PlainMonthDay.from({ month: 12, day: 25 });
function daysUntilChristmas(date = Temporal.Now.plainDateISO()) {
  const xmasThisYear = xmas.toPlainDate(date);
  const xmasNextYear = xmasThisYear.add({ years: 1 });
  return Math.min(xmasThisYear.since(date).days, xmasNextYear.since(date).days);
}

function displayWebsite(transactionData) {
  const { purchasedAt } = transactionData;  // assume this is a Temporal.ZonedDateTime
  const showAd = daysUntilChristmas(purchasedAt) < 30;  // more ergonomic
  // const showAd = daysUntilChristmas(purchasedAt.toPlainDate()) < 30;  // less ergonomic
}

Combining these two principles, any other Temporal method (currently except for toLocaleString) that expects a Calendar instance will also accept a Temporal object that has a calendar property, a property bag containing a calendar field, or an ISO string that can be parsed into any Temporal object that has a calendar property. (A string calendar ID or a CalendarProtocol-satisfying object will also be accepted.). Ditto for TimeZone too.

I assume if TG2 agree with this, the plan is for Justin to propose a PR to change the section "15.4.1 InitializeDateTimeFormat " in the Temporal proposal , it that correct?

https://tc39.es/proposal-temporal/#sec-temporal-initializedatetimeformat

If approved by TG2, one of the Temporal champions will author a PR to make those changes. Unless @sffc you want TG2 to send that PR?

[FrankYFTang]

I still have a hard time to understand this, how about we discuss case 3 and 4 separately first.

For case 4, let's say we have

const xmas = Temporal.PlainMonthDay.from({ month: 12, day: 25 });
const zdt = Temporal.Now.zonedDateTime( "coptic", ""Europe/Moscow" );

why do we need case 4 for

xmas.toLocaleString("ru", {timeZone: zdt.timeZone, calendar:  zdt.calendar});

?

are you saying you need it to be shortern to

xmas.toLocaleString("ru", {timeZone: zdt, calendar:  zdt});

?

[justingrant]

are you saying you need it to be shortern to

xmas.toLocaleString("ru", {timeZone: zdt, calendar:  zdt});

Yes, that's case 4.

Case 3 is this:

xmas.toLocaleString("ru", {timeZone: zdt.toString(), calendar:  zdt.toString()});
xmas.toLocaleString("ru", {timeZone: zdt.toString(), calendar: '2020-01-01'}); // assumes the ISO calendar because no annotation is present.

[FrankYFTang]

Case 3 is this:

xmas.toLocaleString("ru", {timeZone: zdt.toString(), calendar:  zdt.toString()});
xmas.toLocaleString("ru", {timeZone: zdt.toString(), calendar: '2020-01-01'}); // assumes the ISO calendar because no annotation is present.

I cannot understand why we need to support that.

A slightly different topic. Assuming we like to support both case 3 and 4. Could we spec the PR in a way which will only change the step inside toLocaleString but before calling into

3. Let dateFormat be ? Construct(%DateTimeFormat%, « locales, options »).

and not touch anything inside

15.4.1 InitializeDateTimeFormat ( dateTimeFormat, locales, options )

My understanding is you only need to make Temporal.*.prototype.toLocaleString() to handle that, but have no need for the Intl.DateTimeFormat constructor to handle that, right?

The reason I asked that is because any change inside InitializeDateTimeFormat has the risk to increase unnecessary latency for Date.prototype.toLocaleString unnecessarily. And my understanding is you just need to make the Temporal.*.prototype.toLocaleString() to align with other Temporal methods and putting the change inside InitializeDateTimeFormat is more than what we need to do to achieve your goal with unncessary burden to slow down Date.prototype.toLocaleString

[zbraniecki]

I'm not convinced of the value of this extension just yet.

The good news is that this seems like a nice ergonomic addition that is backward compatible and can be added at any time. Which means, we can standardize Temporal, observe how people use it, verify if the lack of (3) or (4) is a perceived papercut, and then add it.

Like Frank, I am also inclined to decouple (3) from (4) for the sake of value proposition evaluation (I'm ok keeping it in a single issue/single proposal/single PR).

[justingrant]

Case 3 is this:

xmas.toLocaleString("ru", {timeZone: zdt.toString(), calendar:  zdt.toString()});
xmas.toLocaleString("ru", {timeZone: zdt.toString(), calendar: '2020-01-01'}); // assumes the ISO calendar because no annotation is present.

I cannot understand why we need to support that.

Sorry, that was a quick, trivial example to illustrate the functionality. A more realistic use case would be single-line formatting of ISO strings into a localized format, without the hassle of creating any intermediate objects.

Here's what I expect to be the most common use of this pattern: formatting a timestamp string while retaining the original offset.

timestamp = '2023-01-07T17:33:21-08:00';
Temporal.Instant.from(timestamp).toLocaleString('en', { timeZone: timestamp, timeZoneName: 'longOffset' }); 

Note that even if we fix this issue, the code above won't run until #683 is fixed too.

[justingrant]

Assuming we like to support both case 3 and 4. Could we spec the PR in a way which will only change the step inside toLocaleString but before calling into

3. Let dateFormat be ? Construct(%DateTimeFormat%, « locales, options »).

@FrankYFTang My top priority is making Temporal APIs internally consistent, so if we only change Temporal.*.p.toLocaleString that's better than nothing! That said, it does seem simpler for both implementers and end-users to make Temporal.*.p.toLocaleString, new Intl.DateTimeFormat, and Date.p.toLocaleString behavior consistent too.

@sffc - do you have an opinion about this?

If we didn't change InitializeDateTimeFormat, then Temporal would have to clone the caller's options object, ensure that calendar and timeZone options are strings, and pass that new object down to Construct(%DateTimeFormat%, .... I'm not sure if that cloning would introduce any issues. @gibson042, do you think this would be a problem?

Also note that if we only change Temporal.*.p.toLocaleString, we'd have to change toLocaleString docs from this:

The locales and options arguments are the same as in the constructor to Intl.DateTimeFormat.

...to something like this:

The locales and options arguments are the same as in the constructor to Intl.DateTimeFormat, except:

• The calendar option accepts any valid input to Temporal.Calendar.from, as long as the resulting object's id represents a built-in calendar's ID.
• The timeZone option accepts any valid input to Temporal.TimeZone.from, as long as the resulting object's id represents a built-in time zone's ID.

Regardless of what we do here in this issue, we may want to change the docs for new Intl.DateTimeFormat to clarify that built-in Temporal.TimeZone and Temporal.Calendar instances are supported, as well as custom calendars/timezones as long as their ids are built-in identifiers.

[justingrant]

any change inside InitializeDateTimeFormat has the risk to increase unnecessary latency for Date.prototype.toLocaleString unnecessarily.

@FrankYFTang If we decide to support cases (3) and (4), then the spec change I had in mind to InitializeDateTimeFormat would only affect the failure case where the calendar or time zone options, when coerced to a string, don't match a built-in calendar or time zone ID.

So I believe that only cases where an exception is thrown today should have a significant change in perf, but you'd be a better judge of this than me. Here's the changes I had in mind:

(calendar option: current Temporal spec)
6. Let _calendar_ be ? GetOption(_options_, *"calendar"*, *"string"*, *undefined*, *undefined*).
7. If _calendar_ is not *undefined*, then
  a. If _calendar_ does not match the Unicode Locale Identifier `type` nonterminal, throw a *RangeError* exception.

(calendar option: proposed)
6. Let _calendar_ be ? Get(_options_, *"calendar"*).
7. If _calendar_ is not *undefined*, then
  a. Let _calendarIdentifier_ be ? ToString(_calendar_).
  b. If IsBuiltinCalendar(_calendarIdentifier_) is *false*, then
    i. Let _calendar_ be ? ToTemporalCalendar(_calendar_).
    ii. Set _calendarIdentifier_ to ? ToString(_calendar_).
    iii. If IsBuiltinCalendar(_calendarIdentifier_) is *false*, throw a *RangeError* exception.
  c. Set _calendar_ to _calendarIdentifier_.

(timeZone option: current Temporal spec)
31. Else,
  a. Set _timeZone_ to ? ToString(_timeZone_).
  b. If IsAvailableTimeZoneName(_timeZone_) is *false*, then
    i. Throw a *RangeError* exception.
  c. Set _timeZone_ to ! CanonicalizeTimeZoneName(_timeZone_).

(timeZone option: proposed)
31. Else,
  a. Let _timeZoneIdentifier_ to ? ToString(_timeZone_).
  b. If IsAvailableTimeZoneName(_timeZoneIdentifier_) is *false*, then
    i. Let _timeZone_ be ? ToTemporalTimeZone(_timeZone_).
    ii. Set _timeZoneIdentifier_ to ? ToString(_timeZone_).
    iii. If IsAvailableTimeZoneName(_timeZoneIdentifier_) is *false*, throw a *RangeError* exception.
  c. Set _timeZone_ to _timeZoneIdentifier_.

Note that a fix to #683 may also need to change the timezone-related spec text above.

[FrankYFTang]

Could you perform these steps on the calendar and timeZone property inside the toLocaleString algorithm BEFORE calling into InitializeDateTimeFormat as a "preprocessing steps" to the options object so InitializeDateTimeFormat will have no need to be changed to support Temporal objects and we will have no need to write additional tests to ensure Date.prototype.toLocaleString accept Temporal objects in the option's calendar and timeZone fields ?

I am not oppose to make Temporal.*.prototype.toLocaleString to accept Temporal objects in calendar/timeZone fields. But I am oppose to make Intl.DateTimeFormat constructor and Date.prototype.toLocaleString to ALSO support them. I view this is an unncessary contamination of feature blow and have the risk as snowballing a limited scope project to larger and larger and out of control.

Notice your proposed change will also cause a huge compatability issue. Consider the following code:

(new Date()).toLocaleString("fr", {calendar: "hello"});

In your proposed change, it will throw RangeError on a method unrelated Temporal but try that on all browsers you can find now.

Notice the string "hello" is true for the statement "match the Unicode Locale Identifier type nonterminal" but will return false on IsBuiltinCalendar(). This is by-design not a spec bug because "match the Unicode Locale Identifier type nonterminal" only require well-form checked.

If TG2 agree to support Temporal objects in calendar/TimeZone field of Temporal.*.prototype.toLocaleString(), then I would like to seek a path that only support such WITHOUT adding any addional support to any methods OTHER THAN Temporal objects. (in specific, make zero impact to Date.prototype.* and Intl.DateTimeFormat constructor). As I point out, it is totally possible that algorithm in Temporal.*.prototype.toLocaleString perform "pre-processing" steps to calendar/timeZone before calling into InitializeDateTimeFormat to achieve that. Temporal spec is already in Stage 3 for a very long time and keep changing as how other Stage 2 spec change. I really do not want to see this Stage 3 spec now start to change the behavior of OTHER objects in ECMA262 and ECMA402.

[sffc]

I think Intl.DateTimeFormat should handle cases 3 and 4. Not just toLocaleString because (1) this is all part of the same ecosystem and (2) I'd rather not have toLocaleString contain much additional logic.

However, I believe that this is going to be fine to add later, because cases 3 and 4 both currently throw exceptions:

new Intl.DateTimeFormat("und", { calendar: "2023-01-09[u-ca=gregory]" })
// VM226:1 Uncaught RangeError: Invalid calendar : 2023-01-09[u-ca=gregory]
//     at new DateTimeFormat (<anonymous>)
//     at <anonymous>:1:1

We can make this case no longer throw in the future if desired.

[justingrant]

Notice your proposed change will also cause a huge compatability issue. Consider the following code:

(new Date()).toLocaleString("fr", {calendar: "hello"});

In your proposed change, it will throw RangeError on a method unrelated Temporal but try that on all browsers you can find now.

Notice the string "hello" is true for the statement "match the Unicode Locale Identifier type nonterminal" but will return false on IsBuiltinCalendar(). This is by-design not a spec bug because "match the Unicode Locale Identifier type nonterminal" only require well-form checked.

I assumed that there was a check somewhere downstream that verified that the calendar was valid. But it looks like there isn't! Unrelated to Temporal, the code below doesn't cause an exception today.

new Intl.DateTimeFormat("en", { calendar: "hello" }).format(new Date());
// => '1/9/2023'

However, the code below does throw:

new Intl.DateTimeFormat("en", { timeZone: "hello" }).format(new Date());
// => RangeError: Invalid time zone specified: hello
new Intl.DateTimeFormat("en", { timeZone: "Asia/Hello" }).format(new Date());
// => RangeError: Invalid time zone specified: Asia/Hello

Is this expected behavior? Why does Intl.DateTimeFormat accept an invalid calendar without throwing? Is there a user benefit, or was this just a bug?

[gibson042]

Yes, that behavior is expected. Intl formatting interfaces generally—but not always—accept any syntactically valid input and perform a best-effort attempt to satisfy it, transforming unknown configuration (especially locale configuration) in the process. For example, on my machine:

(new Intl.DateTimeFormat("sa", { calendar: "hello" })).resolvedOptions().calendar;
// => "gregory"
(new Intl.DateTimeFormat("sa-u-tz-xxx")).resolvedOptions().timeZone;
// => "America/New_York"

(keeping in mind that "tz" is a valid BCP 47 "u" extension key but not currently relevant for Intl.DateTimeFormat)

In contrast, { timeZone: "hello" } and { timeZone: "Asia/Hello" } both fail because InitializeDateTimeFormat validates the value of a timeZone option against the IANA time zone database using IsValidTimeZoneName. And I don't think there's currently a clear guideline or predominant pattern for strict vs. loose validation of options that overlap with locale "u" extension keys, but I would oppose loose validation of time zone on practical grounds alone (e.g., the hazardous consequences of silent transformation).

[sffc]

It's a strange inconsistency, but I think it's harmless and likely intentional; the Intl.DateTimeFormat simply falls back to the default calendar system for the locale if the provided one is not supported, just like it does with other user preferences. It's unsafe to do this with time zones because it could result in a different time of day being displayed.

[justingrant]

It's a strange inconsistency, but I think it's harmless and likely intentional; the Intl.DateTimeFormat simply falls back to the default calendar system for the locale if the provided one is not supported, just like it does with other user preferences. It's unsafe to do this with time zones because it could result in a different time of day being displayed.

Got it. Sounds like my proposed change above would be OK for time zones, but not for calendars. I'd need to think about whether that desired behavior can be achieved at all for calendars. Luckily the use cases where I expect cases (3) and (4) to matter most are time zone cases.

If TG2 agree to support Temporal objects in calendar/TimeZone field of Temporal..prototype.toLocaleString(), then I would like to seek a path that only support such WITHOUT adding any addional support to any methods OTHER THAN Temporal objects. (in specific, make zero impact to Date.prototype. and Intl.DateTimeFormat constructor). As I point out, it is totally possible that algorithm in Temporal.*.prototype.toLocaleString perform "pre-processing" steps to calendar/timeZone before calling into InitializeDateTimeFormat to achieve that.

@gibson042 (or anyone else) - Would it be a problem for Temporal's toLocaleString implementations to shallow-clone the user's options object (and then mutate the calendar and timeZone properties of that cloned object) before passing the options object downstream? Is there precedent for doing this elsewhere in 402 or in ECMAScript overall?

[FrankYFTang]

Is there precedent for doing this elsewhere in 402 or in ECMAScript overall?

See the calling into ToDateTimeOptions for the following https://tc39.es/ecma402/#sup-date.prototype.tolocalestring https://tc39.es/ecma402/#sup-date.prototype.tolocaledatestring https://tc39.es/ecma402/#sup-date.prototype.tolocaletimestring

Can we make a similar ToTemporalOptions AO for Temporal.*.prototype.toLocaleString that resolve the calendar and timeZone according to your desire acceptance criteria from Temporal object and pass that into

4. Let dateFormat be ? Construct(%DateTimeFormat%, « locales, options »).

in toLocaleString?

[justingrant]

See the calling into ToDateTimeOptions for the following https://tc39.es/ecma402/#sup-date.prototype.tolocalestring https://tc39.es/ecma402/#sup-date.prototype.tolocaledatestring https://tc39.es/ecma402/#sup-date.prototype.tolocaletimestring

Can we make a similar ToTemporalOptions AO for Temporal.*.prototype.toLocaleString that resolve the calendar and timeZone

@Ms2ger - in ~~this commit~~ this PR, you removed the call to ToDateTimeOptions from InitializeDateTimeFormat. Do you remember why it was removed? I want to make sure that the approach used in ToDateTimeOptions would be OK for Temporal's toLocaleString too, and that it hasn't been superseded by some other AO.

What's funny is that @sffc asked the same question over 3 years ago... Shane you were a trend-setter! :-)

[Ms2ger]

As I recall, ToDateTimeOptions fills in a bunch of defaults that weren't appropriate to create the new Temporal patterns. The new approach was to read only the actually-provided options into the formatOptions record, and do the defaulting non-observably (creating expandedOptions). Possibly this implies that some properties are Get'ed in a different order, but I think that would have been the maximal extent of the change.

[justingrant]

As I recall, ToDateTimeOptions fills in a bunch of defaults that weren't appropriate to create the new Temporal patterns.

@Ms2ger ToDateTimeOptions is also used by Date.p.toLocale*String methods. Do those uses of ToDateTimeOptions also need be changed?

[anba]

ToDateTimeOptions will be removed, if #709 gets accepted.

[FrankYFTang]

Just to clarify. My mention of "ToDateTimeOptions" is only to answer the question of "Is there precedent for doing this elsewhere in 402 or in ECMAScript overall?"

And the answer is YES by showing current ToDateTimeOptions does so. Would ToDateTimeOptions be removed in https://github.com/tc39/ecma402/pull/709 or not will not change the answer of "precedent" now and in the future.

[sffc]

TG2 discussion: https://github.com/tc39/ecma402/blob/master/meetings/notes-2023-01-12.md#should-intl-apis-match-temporal-interpretation-of-calendar-and-timezone-options-741

Conclusion: "We are not doing any change at moment; I propose to revisit it after Temporal lands at Stage 4 and decide way to go. Put in ES2024, revisit at that point"