Infer type guard => array.filter(x => !!x) should refine Array<T|null> to Array<T>

[danvk]

TypeScript Version: 2.3

Code

const evenSquares: number[] =
    [1, 2, 3, 4]
        .map(x => x % 2 === 0 ? x * x : null)
        .filter(x => !!x);

with strictNullChecks enabled.

Expected behavior:

This should type check. The type of evenSquares should be number[].

Actual behavior:

The type of evenSquares is deduced as (number|null)[], despite the null values being removed via the call to .filter.

[mhegazy]

For filter specifically, https://github.com/Microsoft/TypeScript/issues/7657 would be something we can do here. but that still requires you to write an explicit type guard annotation somewhere, (x: number| null): x is number somewhere. the other part is the compiler figuring out that type guard automatically from the function body, but that is not a trivial, https://github.com/Microsoft/TypeScript/issues/5101 tracks some aspects of that.

[RyanCavanaugh]

Reopening to track this since both other issues now target lib.d.ts changes

[dgreene1]

Subscribing since this is a common pattern I use after a map

[AlexGalays]

Just chiming in to say that this shouldn't be fixed just for Arrays. This is very useful for all custom (container or not) types too.

[dgreene1]

Here's a workaround:

// I wish I could just do the following:
let result = ["", 3,null, undefined].filter(x => x != null);

Instead, you can do this:

// Create this helper function
function isNotNullOrUndefined<T extends Object>(input: null | undefined | T): input is T {
    return input != null;
}
// This actually determines that result is of type (string|number)[]
let result = ["", 3,null, undefined].filter(isNotNullOrUndefined);

I believe that the first snippet doesn't work due to TS not being able to automatically infer that the callback function inside the filter is a type guard. So by explicitly defining the function's return type as input is T then it allows filter to utilize the control flow features of the type checking to discriminate the union.

###################################################

That being said, I hope that we can get this type guard inference into the language. :)

[benschulz]

As I said in https://github.com/Microsoft/TypeScript/issues/10734#issuecomment-351570480, I'm very eager to see type guards inferred and have already implemented it prototypically for a subset of arrow expressions. If you get around to specifying the workings of type guard inference (i.e. the hard part), I'd gladly get involved in the implementation (the easier part).

[scott-ho]

@dgreene1 I trid to write a simple operator to make the guard simpler to use, but failed. Do you have any suggestion?

import { Observable } from 'rxjs';
import { filter } from 'rxjs/operators';

export function isNotNullOrUndefined<T>(input: null | undefined | T): input is T {
  return input != null;
}

export function skipEmpty<T>() {
  return function skipEmptyOperator(source$: Observable<T>) {
    return source$.pipe(filter(isNotNullOrUndefined));
  };
}

@andy-ms @mhegazy could you help to improve the types above?

[Hotell]

@scott-ho https://twitter.com/martin_hotell/status/999707821980647424 you're welcome :) 💃

[dgreene1]

@scott-ho, I'd check out the approach @Hotell shared. I wish I could help you more, but I'm not familiar with rxJs yet. So I'm not really sure what pipe() and filter() are sending you. Since the filter I know is the filter that exists on arrays, and since the snippet above doesn't have filter on an array, then I don't think it's the filter that pertains to this TypeScript issue. I.e. it's not the standard ES5 filter, it's rxJs' filter.

[scott-ho]

@Hotell Thanks for your guidance. It seems your solution only works in v2.8 or above.

And I finally make it works

import { Observable } from 'rxjs';
import { filter } from 'rxjs/operators';

export function isNotNullOrUndefined<T>(input: null | undefined | T): input is T {
  return input != null;
}

export function skipEmpty<T>() {
  return function skipEmptyOperator(source$: Observable<null | undefined | T>) {
    return source$.pipe(filter(isNotNullOrUndefined));
  };
}

[pie6k]

I thought I'll share my solution

export type Empty = null | undefined;

export function isEmpty(value: any): value is Empty {
  return [null, undefined].includes(value);
}

export function removeEmptyElementsFromArray<T>(array: Array<T | Empty>): T[] {
  return array.filter((value) => !isEmpty(value)) as T[];
}

example:

const nums = [2, 3, 4, null] // type is (number | null)[]
const numsWithoutEmptyValues = removeEmptyElementsFromArray(nums); // type is number[]

[samhh]

@pie6k As far as I can tell that's not a solution, that's merely your assertion (as T[]) unsafely narrowing.

[MartinJohns]

@pie6k That [null, undefined].includes(..) will always create a new array, won't it?

Here's a cleaned up version that has no need for type assertions:

function hasValue<T>(value: T | undefined | null): value is T {
    return value !== undefined && value !== null;
}

function removeEmptyElementsFromArray<T>(array: Array<T | undefined | null>): T[] {
    return array.filter(hasValue);
}

[pie6k]

@SamHH it's narrowing, but it's doing it safely as we're removing empty values before

[samhh]

@pie6k It's not safe. You can prove this like so:

export function removeEmptyElementsFromArray<T>(array: Array<T | Empty>): T[] {
  return array as T[];
}

This still compiles just as yours did. It's the assertion that's pleasing the compiler; you can remove the isEmpty function's type guard as it's not doing anything. Your safety is not coming from the type system, but it should do.

I don't think there's any way to generically type guard a negative, which is what your example needs to drop the assertion.

[benschulz]

I believe you're talking past one another. removeEmptyElementsFromArray as posted by @pie6k and @MartinJohns are sound. However, small changes to the code may break soundness without warning because the compiler is blinded by explicitly added type assertions. That's what @SamHH is getting at.

This ~~code~~ issue is about inferring type guards and therefore I have to agree with @SamHH that the code above is not a solution: You're still manually adding type annotations.

[mikeyhew]

Referencing https://github.com/microsoft/TypeScript/issues/18122, because this is what is required for the example code there to work, and I can't comment there because it's closed locked.

[kasperpeulen]

This is quite a clean solution for this problem, using type predicates: http://www.typescriptlang.org/docs/handbook/advanced-types.html#using-type-predicates

function isNotNull<T>(it: T): it is NonNullable<T> {
  return it != null;
}

const nullableNumbers = [1, null];
const nonNullableNumbers = nullableNumbers.filter(isNotNull);

[eduter]

@kasperpeulen, if you make this mistake, it still type-checks:

function isNotNull<T>(it: T): it is NonNullable<T> {
  return it === null; // oops...
}

Using a callback with a type predicate is not much safer than using a type assertion on the return value.

And using an arrow function does not look clean at all:

nullableItems.filter((it): it is NonNullable<typeof it> => it !== null)

[AlexGalays]

This is quite a clean solution for this problem, using type predicates: http://www.typescriptlang.org/docs/handbook/advanced-types.html#using-type-predicates

function isNotNull<T>(it: T): it is NonNullable<T> {
  return it != null;
}

const nullableNumbers = [1, null];
const nonNullableNumbers = nullableNumbers.filter(isNotNull);

This is not a solution to this problem since this problem is all about inferring the type guard.

[kasperpeulen]

I wrote this for people that wonder how they can filter all nulls from an array in a way that typescript understand. If you use this isNotNull as a utility function, then I think just writing nullableNumbers.filter(isNotNull) is quite clean.

However, I totally agree that it would be awesome if typescript can infer any type predicates itself.

[robertmassaioli]

To avoid everybody having to write the same helper functions over and over again I bundled isPresent, isDefined and isFilled into a helper library:

When Typescript bundles this functionality in I'll remove the package.

To use the library:

import { isPresent, isDefined, isFilled } from 'ts-is-present';

arrayWithUndefinedAndNullValues.filter(isPresent)
arrayWithUndefinedValues.filter(isDefined)
arrayWithNullValues.filter(isFilled)

To see why this package works please read the readme of the NPM package.

[pie6k]

Wow! @robertmassaioli this is cool!

function isPresent<T>(t: T | undefined | null | void): t is T {
  return t !== undefined && t !== null;
}

const foo: Array<number | null> = [2,3, null, 4];

const bar = foo.filter(isPresent); // number[]

[jtomaszewski]

Similar to isPresent from ts-is-present library, me and @Smtih just created this utility hasPresentKey fn that helps you filter items down to the ones that have defined non-null item at a[k]. Maybe you'll find it useful.

Source:

export function hasPresentKey<K extends string | number | symbol>(k: K) {
  return function <T, V>(
    a: T & { [k in K]?: V | null }
  ): a is T & { [k in K]: V } {
    return a[k] !== undefined && a[k] !== null;
  };
}

Edit: This, and other similar functions (hasKey hasDefinedKey hasValueAtKey) hopefully will get released soon in the ts-is-present library ( https://github.com/robertmassaioli/ts-is-present/pull/1 ).

[robertmassaioli]

@jtomaszewski That looks like a good addition to ts-is-present please raise a PR against that repository with the code changes and applicable docs and we'll get this out the door. Cheers!

Update from 21/Sep/2020: Okay, it seems that the above functions don't quite typecheck the way that we would like them to. PR in which we are investigating further here: https://github.com/robertmassaioli/ts-is-present/pull/1

[jtomaszewski]

Here's what bitbucket says to me when I'm trying to open a PR to your repo:

Any chance moving it to GH?

[robertmassaioli]

@jtomaszewski I have moved the repository to Github. I want you to get credit in the commits so please feel free to try again there: https://github.com/robertmassaioli/ts-is-present

[graup]

Is this also the issue tracking that I would expect these two lines to be the same?

const values = [4, "hello"] as const;
const result1 = values.filter(val => typeof val === 'number');  // (4 | "hello")[]
const result2 = values.flatMap(val => typeof val === 'number' ? val : []);  // 4[]

I understand the issue is that predicate-like functions aren't inferred as type guards, so any type narrowing that might happen inside it isn't available to the context? In this example, of course I can write is number, but that doesn't work with more complex union types. I'd really like to use the inference (which is already happening anyway).

[AlexGalays]

@graup Seems like the same issue yes.

[SmileJayden]

After es2019, you can use flatMap.

Here is my ts-playground.

const myArr = ['hello', null, 'world', undefined];
const filteredArr = myArr.flatMap((x) => x ? x : []);

Above filteredArr is inferred as string[].

fyi: However, because this approach generates empty array whenever meets nullish value, it has critical performance issue. 😓 I recommend you guys to avoid this approach in production code because of this performance issue.