Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Narrow generic conditional and indexed access return types when checking return statements #56941

Merged
merged 102 commits into from
Nov 6, 2024

Conversation

gabritto
Copy link
Member

@gabritto gabritto commented Jan 3, 2024

Fixes #33912.
Fixes #33014.

Motivation

Sometimes we want to write functions whose return type is picked between different options, depending on the type of a parameter. For instance:

declare const record: Record<string, string>;
declare const array: string[];

function getObject(group) {
    if (group === undefined) {
        return record;
    }
    return array;
}

const arrayResult = getObject("group");
const recordResult = getObject(undefined);

If we want to precisely express this dependency between the return type and the type of nameOrId, we have a few options.
The first one is to use overloads:

declare const record: Record<string, string[]>;
declare const array: string[];

function getObject(group: undefined): Record<string, string[]>;
function getObject(group: string): string[];
function getObject(group: string | undefined): string[] | Record<string, string[]>;
function getObject(group: string | undefined): string[] | Record<string, string[]> {
    if (group === undefined) {
        return record;
    }
    return array;
}

const arrayResult = getObject("group");
const recordResult = getObject(undefined);

However, if you make a mistake in the implementation of the function and return the wrong type, TypeScript will not warn you. For instance, if instead you implement the function like this:

declare const record: Record<string, string[]>;
declare const array: string[];

function getObject(group: undefined): Record<string, string[]>;
function getObject(group: string): string[];
function getObject(group: string | undefined): string[] | Record<string, string[]>;
function getObject(group: string | undefined): string[] | Record<string, string[]> {
    if (!group) { // An empty string is falsy
        return record;
    }
    return array;
}

const badResult = getObject(""); // Type says this returns `string[]`, but actually it returns a record.

then your function implementation doesn't respect the overload signatures, but TypeScript will not error.

The alternative to overloads is to use conditional types, like so:

declare const record: Record<string, string[]>;
declare const array: string[];

function getObject<T extends string | undefined>(group: T):
    T extends string ? string[] : T extends undefined ? Record<string, string[]> : never {
    if (group === undefined) {
        return record; // Error! Type 'Record<string, string[]>' is not assignable to type 'T extends string ? string[] : T extends undefined ? Record<string, string[]> : never'.
    }
    return array; // Error! Type 'string[]' is not assignable to type 'T extends string ? string[] : T extends undefined ? Record<string, string[]> : never'
}

const arrayResult = getObject("group");
const recordResult = getObject(undefined);

However, while everything works out for the callers of getObject, in the implementation TypeScript errors on the return statements, because it compares the type of the return expression to the annotated conditional return type, and Record<string, string[]> is not assignable to T extends undefined ? Record<string, string[]> : never.

Solution: conditional return type narrowing

For this PR, I propose a way of checking return statements that understands cases like above.
The idea is that, for the function above:

function getObject<T extends string | undefined>(group: T):
    T extends string ? string[] : T extends undefined ? Record<string, string[]> : never {
    if (group === undefined) {
        return record;
    }
    return array;
}

when checking the return statement return record, TS will know that group has type undefined. TS will also know that type parameter T corresponds exactly to the type of group. Combining those two pieces of information, TS will know that, inside that branch, the expected return type has to be Record<string, string[]> (or a supertype). Then, instead of checking return record's type against T extends string ? string[] : T extends undefined ? Record<string, string[]> : never, it will check return record's type against the Record<string, string[]> branch of the conditional, i.e. a narrowed version of the conditional type. Then there will be no error on the return statement. In the same manner, when checking the return statement return array, TS will know that group has type string, and that therefore it can check the type of the return statement against the string[] branch of the conditional return type.

For now, we can think of it like this: when we check return statement return record, we see that group has narrowed type undefined. Then, we plug that information back into the return type by instantiating T extends string ? string[] : T extends undefined ? Record<string, string[]> : never with T replaced by undefined.

Restrictions

Conditional types

Reasoning about conditional types is pretty tricky. In general, given a function f(...args): SomeConditionalType whose return type is some (generic) conditional type, we are not able to do the special check proposed above, because it wouldn't be safe.
We need to place some restrictions on what the conditional return type looks like in order for TS to safely analyze it.

We can safely analyze a conditional return type that has the shape T extends A ? AType : T extends B ? BType : never.
This means the conditional type needs to be distributive (i.e. its check type is a naked type parameter T) and have never as its false-most type, and it cannot have infer type parameters (e.g. it cannot be T extends [infer A] ? AType : T extends B ? BType : never).

Intuitively, we can think of this conditional type shape as reflecting the kind of code one would write in the implementation of such a function.

In addition to the previous restrictions, the type parameter constraint has to be a union type, and the extends types of the conditional (A and B above) have to be constituents of the type parameter's union constraint (T above), like so:

function fun<T extends A | B>(param: T): T extends A ? AType : T extends B ? BType : never {
    if (isA(param)) { ... }
    else { ... }
}

This is because, to narrow the return type, we first need to narrow the type of param (more on that below). When we narrow the type of param, in a lot of scenarios, we will start from its type, T, or in this case its type constraint, A | B. Then, we will further narrow that type based on information from control flow analysis, e.g. to pick either A or B (see getNarrowableTypeForReference in checker.ts).

Therefore, in this typical case, narrowing param means we will end up with a type that is either A, B, or a subtype of those. In turn, when we plug this narrowed type back into the conditional return type, this means we will be able to pick a branch of the conditional type and resolve it. e.g. if the narrowed type of param is A, the conditional type will resolve to AType.

These additional restrictions are a heuristic meant to capture the "happy path" of narrowing in TS. For instance, if the type parameter's constraint is not a union, then we might have a case like the below:

function bad<T extends unknown>(x: T): void {
  if (x != undefined) {
    const y: {} = x;
  }
  else {
    const y: null | undefined = x; // Error: `x` is not narrowed here
  }
}

function good<T extends {} | null | undefined>(x: T): void {
  if (x != undefined) {
    const y: {} = x;
  }
  else {
    const y: null | undefined = x; // Works: `x` is narrowed here
  }
}

(Note unknown is conceptually equivalent to {} | null | undefined, but writing Ts constraint as a union instead of unknown makes narrowing work.)

Aside: why never

A common way of trying to write a conditional return type is like the following:

function stringOrNumber<T extends string | number>(param: T): T extends string ? string : number {
  if (typeof param === "string") {
    return "some string";
  }
  return 123;
}

const num = stringOrNumber(123);
const str = stringOrNumber("string");
declare let strOrNum: string | number;
const both = stringOrNumber(strOrNum);

This example works fine and it would be safe for TS to allow that function implementation. However, in general, it is not safe to allow this pattern of conditional return type. Consider this case:

function aStringOrANumber<T extends { a: string } | { a: undefined }>(param: T): T extends { a: string } ? string : number {
  if (typeof param.a === "string") {
    return "some string";
  }
  return 123;
}

const aNum = aStringOrANumber({ a: undefined });
const aStr = aStringOrANumber({ a: "" });
// Bad! Type says `number`, but actually should say `string | number`
const aNotBoth = aStringOrANumber({ a: strOrUndef });

The problem boils down to the fact that, when a conditional return type resolves to its false branch, we can't know if the check type is related or not to the extends type. For the example above, when we plug in { a: undefined } for T in T extends { a: string } ? string : number, then we fall into the false branch of the conditional, which is desired because { a: undefined } does not overlap { a: string }. However, when we plug in { a: string | undefined } for T in T extends { a: string } ? string : number, we fall into the false branch of the conditional, but this is not desired because { a: string | undefined } overlaps { a: string }, and therefore the return type could actually be string.

Resolving a conditional type to its false-most branch of a conditional type doesn't provide TS with enough information to safely determine what the return type should be, and because of that, narrowing a conditional return type requires the false-most branch of the conditional to be never.

Type parameter references

As hinted at above, to narrow a conditional return type, we first need to narrow a parameter, and we need to know that the type of that parameter uniquely corresponds to a type parameter.
Revisiting our example:

function getObject<T extends string | undefined>(group: T):
    T extends string ? string[] : T extends undefined ? Record<string, string[]> : never {
    if (group === undefined) {
        return record;
    }
    return array;
}

To narrow the return type T extends string ? string[] : T extends undefined ? Record<string, string[]> : never, we first narrow the type of group inside the if branch, and then we can use that information to reason about what type T could be replaced with. This only works because the declared type of group is exactly T, and also because there are no other parameters that use type T. So in the following cases, TS would not be able to narrow the return type, because there is no unique parameter to which T is linked:

function badGetObject1<T extends string | undefined>(group: T, someOtherParam: T):
    T extends string ? string[] : T extends undefined ? Record<string, string[]> : never {
    ...
}

function badGetObject2<T extends string | undefined>(group: T, options: { a: number, b: T }):
    T extends string ? string[] : T extends undefined ? Record<string, string[]> : never {
    ...
}

Indexed access types

The reasoning explained above for conditional types applies in a similar manner to indexed access types that look like this:

interface F {
  "t": number,
  "f": boolean,
}

function depLikeFun<T extends "t" | "f">(str: T): F[T] {
  if (str === "t") {
    return 1;
  } else {
    return true;
  }
}

depLikeFun("t"); // has type number
depLikeFun("f"); // has type boolean

So cases like this, where the return type is an indexed access type with a type parameter index, are also supported by this PR. The thinking is similar: in the code above, when we are in the if (str === "t") { ... } branch, we know str has type "t", and we can plug that information back into the return type F[T] which resolves to type number, and similarly for the else branch.

Implementation

The implementation works roughly like this:
When checking a return statement expression:

  • We check if the type of the expression is assignable to the return type. If it is, good, nothing else needs to be done. This makes sure we don't introduce new errors to existing code.
  • If the type of the expression is not assignable to the return type, we might need to narrow the return type and check again. We proceed to attempt narrowing the return type.
  • We check if the return type has the right shape, i.e. it has a shape of either SomeType[T] or T extends A ? AType : T extends B ? BType : never, and what parameters the type parameters are uniquely linked to, among other requirements. If any of those requirements is not met, we don't continue with narrowing.
  • For every type parameter that is uniquely linked to a parameter, we obtain its narrowed type.
    • Say we have type parameter T that is uniquely linked to parameter param. To narrow the return type, we first need to obtain the narrowed type for param at the return statement position. Because, in the source code, there might not be an occurrence of param at the return statement position, we create a synthetic reference to param at that position and obtain its narrowed type via regular control-flow analysis. We then obtain a narrowed type N for param. (If we don't, we'll just ignore that type parameter).
  • Once we have the narrowed type for each type parameter, we need to plug that information back into the return type. We do this by instantiating the return type with a substitution type. For instance, if the return type is T extends A ? AType : T extends B ? BType : never, T is linked to param, and param has narrowed type N, we will instantiate the return type with T replaced by T & N (as a substitution type).
  • Once we obtain this narrowed return type, we get the type of the return expression, this time contextually checked by the narrowed return type, and then check if this type is assignable to the narrowed return type.

Instantiation

As mentioned above, the process of narrowing a return type is implemented as instantiating that return type with the narrowed type parameters replaced by substitution types. Substitution types can be thought of as T & A, where T is the base type and A the constraint type. There are a few changes made to instantiation to make this work:

  • There is now a new kind of substitution type, indicated by the ObjectFlags.IsNarrowingType flag, which corresponds to substitution types created by return type narrowing.
  • These narrowing substitution types are handled in a special way by conditional type instantiation functions, i.e. getConditionalTypeInstantiation, and getConditionalType.
  • getConditionalTypeInstantiation is responsible for distributing a conditional type over its check type. When instantiating a distributive conditional type in getConditionalTypeInstantiation, if the conditional's check type is a substitution type like T & (A | B), the usual logic would not distribute over this type, because it's a substitution type and not a union type. So, for distribution to happen, we have to take apart the T & (A | B) into (T & A) | (T & B), and distribute over that.
  • The other special thing that we have to do in getConditionalTypeInstantiation is to take the intersection of the distribution result, as opposed to the union. This is because, if we narrow a type parameter T to A | B, and we have a conditional return type T extends A ? R1 : T extends B ? R2 : T extends C ? R3 : never, then we don't know which branch of the conditional return to pick, if branch T extends A ? R1, or branch T extends B ? R2, so we have to check whether the return expression's type is assignable to both, i.e. assignable to R1 & R2.
  • Validating whether the conditional type has the right shape to be narrowed happens on-demand in getConditionalTypeInstantiation (and also at first as an optimization in checkReturnExpression when we're deciding whether to narrow the return type). This is because, as we instantiate a type, we may produce new conditional types that we need to then decide are safe to narrow or not (see nested types case in dependentReturnType6.ts test).

Conditional expression checking

To support conditional expression checking in return statements, this PR changes how we check a conditional expression in a return statement. Before this PR, when checking a return statement return cond ? exp1 : exp2, we obtain the type of the whole expression cond ? exp1 : exp2, and then compare that type to the return type. With this PR, we now separately check each branch: we first obtain the type of exp1, and compare that type to the return type, then obtain the type of exp2 and compare that type to the return type. This allows us to properly check a conditional expression when return type narrowing is needed.

This a breaking change, and the only change that affects existing code, but this change finds bugs. Analysis of extended tests changes: #56941 (comment).
This change also slightly affects performance because we do more checks. Latest perf results here: #56941 (comment).

Performance results

This feature is opt-in. Currently, virtually no code has functions whose return types are conditional or indexed access types that satisfy the restrictions above, so no code goes down the new code path for narrowing return types. This means for existing code, there's no performance impact from the return type narrowing. The only current performance impact is from the conditional expression checking change (see a version of this PR without the change for conditional expressions: #60268 (comment)).

Assessing the performance of the new code path is tricky, as there are no baselines. The existing alternative to conditional return types is to use overloads. In one scenario I tested, checking a function written with conditional return types with this PR took ~+16% check time compared to the same function using overloads and the main branch. However, that's for checking the function declaration. In a different scenario where I included a lot of function calls though, the version with conditional return types + this PR took ~-15% compared to overloads + main branch. So I'd say the performance is acceptable, especially considering you get stronger checks when using conditional return types, and also only a small number of functions in a codebase should be written using this feature.

Unsupported things

  • Inference:
    TS will not infer a conditional return type or an indexed access type for any function or expression. Inferring such a type is more complicated than checking, and inferring such a type could also be surprising for users. This is out of scope.

  • Contextually-typed anonymous functions:

type GetObjectCallback = <T extends string | undefined>(group: T) => T extends string ? string[] : T extends undefined ? Record<string, string[]> : never;

const getObjectBad1: GetObjectCallback =
    (group) => { return group === undefined ? record : array }; // Error

declare function outerFun(callback: GetObjectCallback);
outerFun((group) => { return group === undefined ? record : array }); // Error

This is because, if your function does not have an explicitly annotated return type, we will infer one from the returns.

  • Detection of link between parameter and type parameter in more complicated scenarios:
// All cases below are not recognized
// Type alias
type Id<X> = X;
function f2<T extends boolean>(arg: Id<T>): T extends true ? string : T extends false ? number : never {
  if (arg) {
    return "someString";
  }
  return 123;
}

// Property type
function f3<T extends boolean>(arg: { prop: T }): T extends true ? string : T extends false ? number : never {
  if (arg.prop) {
    return "someString";
  }
  return 123;
}

// Destructuring
function f4<T extends boolean>({ arg }: { arg: T }): T extends true ? string : T extends false ? number : never {
  if (arg.prop) {
    return "someString";
  }
  return 123;
}

// Combinations of the above, e.g.:
type Opts<X> = { prop: X };
function f5<T extends boolean>(arg: Opts<T>): T extends true ? string : T extends false ? number : never {
  if (arg.prop) {
    return "someString";
  }
  return 123;
}

This could be supported in the future.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Jan 3, 2024
@gabritto
Copy link
Member Author

gabritto commented Jan 3, 2024

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 3, 2024

Heya @gabritto, I've started to run the regular perf test suite on this PR at a3a54ce. You can monitor the build here.

Update: The results are in!

@typescript-bot

This comment was marked as outdated.

@gabritto
Copy link
Member Author

gabritto commented Jan 3, 2024

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 3, 2024

Heya @gabritto, I've started to run the regular perf test suite on this PR at 1b7489f. You can monitor the build here.

Update: The results are in!

if (!Meteor.settings) {
return undefined; // Error
~~~~~~
!!! error TS2322: Type 'undefined' is not assignable to type 'HelperCond<I, string, T | undefined, RegExp, SettingComposedValue<T>[]>'.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's too bad we don't give any elaboration here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I don't really know how we could provide more feedback when we can't narrow a return type. It's already not 100% clear whether the user is trying to narrow the return type or not, and even when we think they are, I can't think of a good mechanism to explain why we didn't narrow.

Copy link
Member

@andrewbranch andrewbranch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent write-up. I have a couple questions.

@@ -19081,14 +19098,17 @@ export function createTypeChecker(host: TypeCheckerHost): TypeChecker {
if (checkType === wildcardType || extendsType === wildcardType) {
return wildcardType;
}
const effectiveCheckType = forNarrowing && isNarrowingSubstitutionType(checkType)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need both forNarrowing and to check the flags of the check type?

Copy link
Member Author

@gabritto gabritto Oct 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

forNarrowing = true is passed by getConditionalTypeInstantiation when we validate the conditional type is safe to narrow. It could be that the check type is a narrowing substitution type, but we don't want to narrow the conditional type because it has an invalid shape.

I think I did this because it was inconvenient and a bit confusing for me to call the conditional type validation function from inside getConditionalType every time we loop to a new conditional type. I found it simpler to validate the whole conditional type in getConditionalTypeInstantiation and have getConditionalType loop on that type.

(t: Type) =>
getConditionalType(
root,
prependTypeMapping(checkType, getSubstitutionType(narrowingBaseType, t, /*isNarrowed*/ true), newMapper),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the changes here aren't mentioned in your implementation notes. Why do we need custom logic for distribution when we're narrowing?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question.

We are using substitution types (which can be thought of as T & A, where T is the base type and A the constraint) to convey a type parameter's narrowing. When instantiating a distributive conditional type in getConditionalTypeInstantiation, if the conditional's check type is a substitution type like T & (A | B), the old logic would not distribute over this type, because it's a substitution type and not a union type. So, for distribution to happen, we have to take apart the T & (A | B) into (T & A) | (T & B), and distribute over that.

The other special thing that we have to do is to take the intersection of the distribution result, as opposed to the union. This is because, if we narrow a type parameter T to A | B, and we have a conditional return type T extends A ? R1 : T extends B ? R2 : T extends C ? R3 : never, then we don't know which branch of the conditional return to pick, if branch T extends A ? R1, or branch T extends B ? R2, so we have to check whether the return expression's type is assignable to both, i.e. assignable to R1 & R2.

A semi-related note is that validating whether the conditional type has the right shape to be narrowed happens on-demand (and also at first as an optimization in checkReturnExpression when we're deciding whether to narrow the return type). This is because, as we instantiate a type, we may produce new conditional types that we need to then decide are safe to narrow or not (see nested types case in dependentReturnType6.ts test).

There's a bunch of details I haven't included in the implementation notes because I don't have a full list of them 😅. I'll add those.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to always normalize union-containing substitution types into unions of substitution types before distributing?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't thought about it, but I assume it would break something somewhere in weird ways.

Copy link
Contributor

@Andarist Andarist Oct 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

something akin to that could help with some NoInfer problems: #60271 (comment) :p

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have this vague sense that it would be nice if the conditional-narrowing-specific logic didn’t affect the implementation of getConditionalTypeInstantiation, but even if it always normalized substitution types, I don’t have a suggestion for how to extract mapTypeToIntersection, so I guess it’s moot.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

something akin to that could help with some NoInfer problems: #60271 (comment) :p

I think it's worth investigating what would happen if we normalize substitution types with union constraints like we do for intersections with unions, especially if it fixes some problems. But I'll leave that for the future.

@gabritto gabritto requested a review from sandersn October 30, 2024 18:40
Copy link
Member

@weswigham weswigham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one question and one small suggestion - this looks good~

// A narrowable conditional type is one that has the following shape:
// `T extends A ? TrueBranch<T> : FalseBranch<T>`, in other words:
// (0) The conditional type is distributive;
// (1) The conditional type has no `infer` type parameters;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why's that? Is something like

function f<T extends [number, string] | [string, number]>(arg: T): T extends [number, infer U] ? U : never {
  if (typeof arg[0] === "number") {
    return arg[1];
  }
  throw new Error("Got non numeric tuple");
}

for some reason not feasible to check? Is it just because we'd have to do inference between the true branch type and the candidate return type to fill in the infer type variable, and that's not implemented yet? Because I could equivalently write

function f<T extends [number, U] | [string, number], U extends string>(arg: T): T extends [number, U] ? U : never {
  if (typeof arg[0] === "number") {
    return arg[1];
  }
  throw new Error("Got non numeric tuple");
}

which afaik is valid under these rules. I think all infer type parameters in types satisfying the rest of these rules can be rewritten like this pretty easily, which seems to imply the infer itself aughta work, but I could be wrong.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically I don't know how inference would work safely between the substitution type I'm using to represent a narrowed type parameter and the infer type parameters, given how inference works today.

In T extends [number, infer U] ? U : never, if we instantiate T with substitution type T & [number, string], we would infer from T & [number, string] to [number, infer U].
Now, we can't simply infer string for U, because it would not be sound to narrow the return type to string:
the caller can specify that T is [number, "foo"] and then U is inferred to be "foo", not string.
We'd have to infer something like T[1] & string or simply T[1] for U for this to be safe, and I think we don't have the ability to do that.

src/compiler/checker.ts Show resolved Hide resolved
@gabritto
Copy link
Member Author

@typescript-bot test it

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 31, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top400 ✅ Started 👀 Results
user test this ✅ Started 👀 Results
run dt ✅ Started ✅ Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@gabritto Here are the results of running the user tests with tsc comparing main and refs/pull/56941/merge:

Something interesting changed - please have a look.

Details

adonis-framework

/mnt/ts_downloads/_/m/adonis-framework/tsconfig.json

  • [NEW] error TS2322: Type 'null' is not assignable to type 'string'.
    • /mnt/ts_downloads/_/m/adonis-framework/node_modules/adonis-framework/src/Request/index.js(600,75)

puppeteer

packages/puppeteer-core/tsconfig.json

@typescript-bot
Copy link
Collaborator

Hey @gabritto, the results of running the DT tests are ready.

Everything looks the same!

You can check the log here.

@typescript-bot
Copy link
Collaborator

@gabritto
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 31 34 🔻+3 (+ 9.68%) ~ ~ p=0.001 n=6
Symbols 62,340 62,363 +23 (+ 0.04%) ~ ~ p=0.001 n=6
Types 50,379 50,395 +16 (+ 0.03%) ~ ~ p=0.001 n=6
Memory used 192,901k (± 0.09%) 195,406k (± 0.96%) +2,505k (+ 1.30%) 192,975k 196,677k p=0.031 n=6
Parse Time 1.31s (± 0.92%) 1.31s (± 0.84%) ~ 1.29s 1.32s p=0.432 n=6
Bind Time 0.72s 0.72s ~ ~ ~ p=1.000 n=6
Check Time 9.72s (± 0.56%) 9.78s (± 0.30%) ~ 9.75s 9.82s p=0.087 n=6
Emit Time 2.71s (± 1.45%) 2.74s (± 0.48%) ~ 2.72s 2.76s p=0.141 n=6
Total Time 14.47s (± 0.62%) 14.55s (± 0.23%) ~ 14.52s 14.60s p=0.090 n=6
angular-1 - node (v18.15.0, x64)
Errors 33 37 🔻+4 (+12.12%) ~ ~ p=0.001 n=6
Symbols 947,886 947,934 +48 (+ 0.01%) ~ ~ p=0.001 n=6
Types 410,840 410,955 +115 (+ 0.03%) ~ ~ p=0.001 n=6
Memory used 1,224,607k (± 0.00%) 1,226,060k (± 0.00%) +1,453k (+ 0.12%) 1,226,012k 1,226,117k p=0.005 n=6
Parse Time 6.65s (± 0.85%) 6.66s (± 0.71%) ~ 6.60s 6.72s p=0.809 n=6
Bind Time 1.88s (± 0.27%) 1.89s (± 0.22%) +0.01s (+ 0.44%) 1.89s 1.90s p=0.022 n=6
Check Time 31.90s (± 0.60%) 31.97s (± 0.36%) ~ 31.81s 32.13s p=0.471 n=6
Emit Time 15.24s (± 0.31%) 15.18s (± 0.41%) ~ 15.07s 15.25s p=0.078 n=6
Total Time 55.68s (± 0.36%) 55.69s (± 0.36%) ~ 55.45s 55.90s p=0.936 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,495,026 2,495,029 +3 (+ 0.00%) ~ ~ p=0.001 n=6
Types 908,645 908,650 +5 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,307,202k (± 0.00%) 2,313,900k (± 0.00%) +6,697k (+ 0.29%) 2,313,879k 2,313,936k p=0.005 n=6
Parse Time 9.33s (± 0.40%) 9.33s (± 0.22%) ~ 9.29s 9.35s p=1.000 n=6
Bind Time 2.14s (± 0.19%) 2.14s (± 0.35%) ~ 2.13s 2.15s p=1.000 n=6
Check Time 75.29s (± 0.51%) 74.90s (± 0.36%) ~ 74.41s 75.13s p=0.173 n=6
Emit Time 0.28s 0.29s (± 2.58%) 🔻+0.01s (+ 4.17%) 0.28s 0.30s p=0.009 n=6
Total Time 87.04s (± 0.47%) 86.66s (± 0.32%) ~ 86.16s 86.92s p=0.173 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,258,087 1,258,781 +694 (+ 0.06%) ~ ~ p=0.001 n=6
Types 266,235 266,471 +236 (+ 0.09%) ~ ~ p=0.001 n=6
Memory used 2,422,835k (± 0.01%) 2,425,817k (± 0.01%) +2,983k (+ 0.12%) 2,425,688k 2,426,174k p=0.005 n=6
Parse Time 5.21s (± 1.05%) 5.20s (± 0.53%) ~ 5.17s 5.25s p=0.748 n=6
Bind Time 1.92s (± 0.47%) 1.95s (± 0.39%) +0.03s (+ 1.48%) 1.94s 1.96s p=0.004 n=6
Check Time 35.50s (± 0.35%) 35.59s (± 0.16%) ~ 35.51s 35.65s p=0.298 n=6
Emit Time 3.04s (± 0.65%) 3.02s (± 1.22%) ~ 2.96s 3.06s p=0.573 n=6
Total Time 45.69s (± 0.31%) 45.74s (± 0.17%) ~ 45.65s 45.84s p=0.575 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,258,087 1,258,781 +694 (+ 0.06%) ~ ~ p=0.001 n=6
Types 266,235 266,471 +236 (+ 0.09%) ~ ~ p=0.001 n=6
Memory used 2,619,643k (±11.18%) 2,622,085k (±11.19%) +2,442k (+ 0.09%) 2,500,959k 3,221,358k p=0.045 n=6
Parse Time 6.61s (± 2.23%) 6.59s (± 1.84%) ~ 6.52s 6.83s p=0.376 n=6
Bind Time 2.16s (± 1.98%) 2.20s (± 4.21%) ~ 2.10s 2.37s p=0.423 n=6
Check Time 43.37s (± 0.45%) 43.39s (± 0.36%) ~ 43.24s 43.67s p=0.810 n=6
Emit Time 3.61s (± 3.02%) 3.52s (± 3.65%) ~ 3.43s 3.77s p=0.149 n=6
Total Time 55.77s (± 0.61%) 55.72s (± 0.47%) ~ 55.53s 56.20s p=0.936 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 261,754 262,180 +426 (+ 0.16%) ~ ~ p=0.001 n=6
Types 106,477 106,602 +125 (+ 0.12%) ~ ~ p=0.001 n=6
Memory used 438,797k (± 0.02%) 439,725k (± 0.01%) +929k (+ 0.21%) 439,660k 439,785k p=0.005 n=6
Parse Time 3.53s (± 1.15%) 3.54s (± 0.39%) ~ 3.52s 3.56s p=0.808 n=6
Bind Time 1.32s (± 0.48%) 1.31s (± 0.79%) -0.01s (- 1.01%) 1.29s 1.32s p=0.028 n=6
Check Time 18.90s (± 0.40%) 18.92s (± 0.17%) ~ 18.89s 18.97s p=1.000 n=6
Emit Time 1.54s (± 1.68%) 1.51s (± 1.17%) ~ 1.49s 1.54s p=0.099 n=6
Total Time 25.29s (± 0.32%) 25.27s (± 0.16%) ~ 25.23s 25.34s p=0.520 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 68 70 +2 (+ 2.94%) ~ ~ p=0.001 n=6
Symbols 225,919 226,060 +141 (+ 0.06%) ~ ~ p=0.001 n=6
Types 94,415 94,488 +73 (+ 0.08%) ~ ~ p=0.001 n=6
Memory used 371,091k (± 0.01%) 371,593k (± 0.01%) +502k (+ 0.14%) 371,534k 371,643k p=0.005 n=6
Parse Time 2.90s (± 0.42%) 2.90s (± 1.39%) ~ 2.86s 2.96s p=0.560 n=6
Bind Time 1.58s (± 1.18%) 1.57s (± 0.77%) ~ 1.56s 1.59s p=0.742 n=6
Check Time 16.36s (± 0.34%) 16.48s (± 0.28%) +0.12s (+ 0.73%) 16.41s 16.55s p=0.008 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 20.83s (± 0.31%) 20.95s (± 0.40%) +0.12s (+ 0.57%) 20.85s 21.09s p=0.020 n=6
vscode - node (v18.15.0, x64)
Errors 3 10 🔻+7 (+233.33%) ~ ~ p=0.001 n=6
Symbols 3,129,928 3,138,510 +8,582 (+ 0.27%) ~ ~ p=0.001 n=6
Types 1,078,947 1,082,406 +3,459 (+ 0.32%) ~ ~ p=0.001 n=6
Memory used 3,219,514k (± 0.03%) 3,228,901k (± 0.00%) +9,388k (+ 0.29%) 3,228,704k 3,229,028k p=0.005 n=6
Parse Time 14.07s (± 0.62%) 14.04s (± 0.35%) ~ 13.95s 14.09s p=0.378 n=6
Bind Time 4.84s (±14.62%) 4.57s (± 2.98%) ~ 4.47s 4.78s p=0.936 n=6
Check Time 86.30s (± 4.17%) 85.18s (± 0.46%) ~ 84.77s 85.73s p=0.575 n=6
Emit Time 26.78s (± 2.23%) 27.95s (± 2.06%) 🔻+1.17s (+ 4.38%) 27.33s 28.69s p=0.005 n=6
Total Time 131.99s (± 2.65%) 131.74s (± 0.58%) ~ 130.83s 132.83s p=0.471 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 287,028 287,028 ~ ~ ~ p=1.000 n=6
Types 116,316 116,316 ~ ~ ~ p=1.000 n=6
Memory used 438,405k (± 0.02%) 438,641k (± 0.02%) +236k (+ 0.05%) 438,550k 438,767k p=0.008 n=6
Parse Time 4.05s (± 0.88%) 4.06s (± 0.96%) ~ 4.01s 4.12s p=1.000 n=6
Bind Time 1.73s (± 1.57%) 1.75s (± 0.98%) ~ 1.73s 1.78s p=0.123 n=6
Check Time 18.54s (± 0.26%) 18.51s (± 0.57%) ~ 18.41s 18.66s p=0.574 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 24.33s (± 0.36%) 24.32s (± 0.45%) ~ 24.17s 24.46s p=0.936 n=6
xstate-main - node (v18.15.0, x64)
Errors 3 4 🔻+1 (+33.33%) ~ ~ p=0.001 n=6
Symbols 543,130 543,130 ~ ~ ~ p=1.000 n=6
Types 181,889 181,891 +2 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 485,472k (± 0.02%) 486,526k (± 0.01%) +1,054k (+ 0.22%) 486,450k 486,656k p=0.005 n=6
Parse Time 4.19s (± 0.38%) 4.19s (± 0.54%) ~ 4.16s 4.21s p=0.807 n=6
Bind Time 1.46s (± 0.72%) 1.47s (± 0.93%) ~ 1.45s 1.49s p=0.160 n=6
Check Time 23.82s (± 1.20%) 23.83s (± 0.29%) ~ 23.77s 23.95s p=0.173 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 29.47s (± 1.03%) 29.48s (± 0.27%) ~ 29.41s 29.62s p=0.228 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@gabritto Here are the results of running the top 400 repos with tsc comparing main and refs/pull/56941/merge:

Something interesting changed - please have a look.

Details

alibaba/formily

29 of 38 projects failed to build with the old tsc and were ignored

packages/grid/tsconfig.json

packages/grid/tsconfig.build.json

antvis/G2

2 of 3 projects failed to build with the old tsc and were ignored

tsconfig.json

cheeriojs/cheerio

2 of 3 projects failed to build with the old tsc and were ignored

tsconfig.typedoc.json

  • error TS2322: Type 'boolean' is not assignable to type 'string | number | Document | Element | CDATA | Text | Comment | ProcessingInstruction | ChildNode[] | ... 6 more ... | undefined'.

GrapesJS/grapesjs

1 of 3 projects failed to build with the old tsc and were ignored

packages/core/tsconfig.json

honojs/hono

7 of 8 projects failed to build with the old tsc and were ignored

tsconfig.json

ionic-team/stencil

39 of 42 projects failed to build with the old tsc and were ignored

scripts/tsconfig.json

jupyterlab/jupyterlab

46 of 59 projects failed to build with the old tsc and were ignored

examples/filebrowser/src/tsconfig.json

examples/console/src/tsconfig.json

microsoft/vscode

5 of 53 projects failed to build with the old tsc and were ignored

src/tsconfig.tsec.json

src/tsconfig.monaco.json

nextauthjs/next-auth

22 of 44 projects failed to build with the old tsc and were ignored

packages/adapter-pg/tsconfig.json

tailwindlabs/headlessui

2 of 5 projects failed to build with the old tsc and were ignored

packages/@headlessui-vue/tsconfig.json

vuejs/devtools-v6

7 of 8 projects failed to build with the old tsc and were ignored

packages/api/tsconfig.json

vuejs/vue

7 of 8 projects failed to build with the old tsc and were ignored

tsconfig.json

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
7 participants