class-transformer icon indicating copy to clipboard operation
class-transformer copied to clipboard

question: defaultMetadataStorage is no more available in 0.3.2

Open Arnaud-Dev-Nodejs opened this issue 4 years ago • 37 comments

Description

In the previous version it was possible to access to the defaultMetadataStorage which was useful to create custom transformer.

for example : Minimal code-snippet showcasing the problem


import { TransformOptions } from 'class-transformer';
import { defaultMetadataStorage } from 'class-transformer/storage/'; <== cannot find module anymore

export function doSplit(separator: string | RegExp = ' ') {
  return (value: any) => {
    return value !== undefined && value !== null
      ? String(value).split(separator)
      : [];
  };
}
export function Split(
  separator: string | RegExp,
  options: TransformOptions = {},
): PropertyDecorator {
  const transformFn = doSplit(separator);

  return function (target: any, propertyName: string | symbol): void {
    defaultMetadataStorage.addTransformMetadata({
      target: target.constructor,
      propertyName: propertyName as string,
      transformFn,
      options,
    });
  };
}

Expected behavior

Get the defaultMetadataStorage !

Actual behavior

"Cannot find module 'class-transformer/storage' or its corresponding type declarations."

Arnaud-Dev-Nodejs avatar Jan 19 '21 10:01 Arnaud-Dev-Nodejs

You should be able to reach it from class-transformer/MetadataStorage.ts, but keep in mind that this is an internal class, and can break anytime.

NoNameProvided avatar Jan 19 '21 11:01 NoNameProvided

Hi, we used defaultMetadataStorage to extract all applied validations so we could construct our OpenAPI schema on the fly. I understand that this is a "private" class of sorts, but we're ready to adapt to any changes, as there is no other way for us to automate OpenAPI generation.

edit 1: Strangely, source code doesn't seem to be changed, but imports don't work the same way. Could it be because of changes in tsconfig.json?

edit 2: Found a workaround, @Arnaud-Dev-Nodejs you may want to try this. import { defaultMetadataStorage } from 'class-transformer/cjs/storage';

igoreso avatar Jan 27 '21 08:01 igoreso

Looks like new release changed the npm library definitions to use cjs and other formats to support library package formats. Looks like this is also breaking the package class-validate-jsonschema since it’s also referencing defaultMetadataStorage...

mlakmal avatar Jan 27 '21 13:01 mlakmal

Strangely, source code doesn't seem to be changed, but imports don't work the same way. Could it be because of changes in tsconfig.json?

Yes, we move to the universal package format for each TypeStack package. However, this should not break your workflow as your build tool should be able to understand this as we reference each entry point properly. https://github.com/typestack/class-transformer/blob/ab093a355be420978273b675d3bbd3a520d1918d/package.json#L9-L12

Can I ask what build tools you use @Igoreso?

NoNameProvided avatar Feb 14 '21 15:02 NoNameProvided

Trying @Igoreso 's workaround produces this error:

Could not find a declaration file for module 'class-transformer/cjs/storage'. './node_modules/class-transformer/cjs/storage.js' implicitly has an 'any' type.
  Try `npm i --save-dev @types/class-transformer` if it exists or add a new declaration (.d.ts) file containing `declare module 'class-transformer/cjs/storage';`ts(7016)

I am also not able to see why the "storage" module has vanished from the typescript detection. maybe only the index is not exposed, and defaultMetadataStorage should just be re-exported from the index module? maybe named so it wont break with future releases?

PhilippMolitor avatar Feb 16 '21 22:02 PhilippMolitor

maybe only the index is not exposed, and defaultMetadataStorage should just be re-exported from the index module?

It won't be re-exported, because it's an internal part of the lib and we don't support its API officially. I won't actively limit the access to it, but it will be never included in the barrel export to indicate that is not an official API.

so it wont break with future releases?

It's an internal part of the lib, it will break with future releases.

NoNameProvided avatar Feb 20 '21 19:02 NoNameProvided

What we can do here is update the project config if needed to allow importing it directly with every build tool from its direct path. For that, I will need what build tool cannot find it.

NoNameProvided avatar Feb 20 '21 19:02 NoNameProvided

Nestjs :)

In fact, nobody needs to access your internal interface. The only thing we want is the ability to add dynamic Transformer.

Arnaud-Dev-Nodejs avatar Feb 22 '21 15:02 Arnaud-Dev-Nodejs

Nestjs :)

In fact, nobody needs to access your internal interface. The only thing we want is the ability to add dynamic Transformer.

That is misleading.
I'd argue that everybody who has an advanced use-case absolutely needs access to the internal interface because there is no other way to do that.

E.g. we need access to the column-mapping somehow. Of course, we would also prefer a public, stable and documented way to do this (instead of accessing internal private parts, that can change without notice between releases).
But we do understand that this is an advanced use-case and we don't expect the library authors to invest lots of effort, just to cover exotic use-cases.

tmtron avatar Feb 22 '21 15:02 tmtron

same problem here, updating to 0.3.2 breaks the import. using require syntax instead of imports works though. Node version 14

// import { defaultMetadataStorage as classTransformerDefaultMetadataStorage, } from "class-transformer/storage";
// eslint-disable-next-line @typescript-eslint/no-var-requires, @typescript-eslint/no-require-imports
const classTransformerDefaultMetadataStorage = require(`class-transformer/storage`);

nolazybits avatar Mar 17 '21 21:03 nolazybits

You can also import the ES module as import { defaultMetadataStorage } from 'class-transformer/esm5/storage' and then provide types separately with a .d.ts declaration like

declare module 'class-transformer/esm5/storage' {
  import type { MetadataStorage } from 'class-transformer/types/MetadataStorage';

  export const defaultMetadataStorage: MetadataStorage;
}

This is a bit of a hassle though: exposing defaultMetadataStorage or another API for accessing metadata objects would make it much easier to develop libraries that integrate with class-transformer. Note that metadata object types are already part of the public API.

epiphone avatar Mar 20 '21 06:03 epiphone

Just wondering @NoNameProvided why you wouldn't want to expose it? There are needs it seems from this thread.

One I can contribute to is we are testing our Models to make sure the dto (domain transfer object) don't change Here is the class to do those validation

/* eslint-disable @typescript-eslint/ban-types */
import { ExcludeOptions, ExposeOptions } from "class-transformer";
import { ExcludeMetadata, ExposeMetadata, TypeMetadata } from "class-transformer";
// import { defaultMetadataStorage as classTransformerDefaultMetadataStorage } from "class-transformer/types/storage";
const classTransformerDefaultMetadataStorage = require(`class-transformer/storage`);
import { getMetadataStorage, ValidationOptions, ValidationTypes } from "class-validator";
import { ConstraintMetadata } from "class-validator/types/metadata/ConstraintMetadata";
import { ValidationMetadata } from "class-validator/types/metadata/ValidationMetadata";

export class ValidationConstraintUtils
{
    public static TestExposeMetadata(target: Function, propertyName?: string, options: ExposeOptions = {}): void
    {
        // eslint-disable-next-line @typescript-eslint/ban-ts-comment
        // @ts-ignore
        const exposeMetadata: ExposeMetadata = classTransformerDefaultMetadataStorage.findExposeMetadata(target, propertyName);
        expect(exposeMetadata).toBeDefined();
        expect(exposeMetadata.options).toEqual(options);
    }

    public static TestNotExposeMetadata(target: Function, propertyName?: string, options: ExcludeOptions = {}): void
    {
        // eslint-disable-next-line @typescript-eslint/ban-ts-comment
        // @ts-ignore
        const exposeMetadata: ExposeMetadata | undefined = classTransformerDefaultMetadataStorage.findExposeMetadata(target, propertyName);
        expect(exposeMetadata).toBeUndefined();
    }

    // eslint-disable-next-line @typescript-eslint/ban-ts-comment
    // @ts-ignore
    public static TestExcludeMetadata(target: Function, propertyName?: string, options: ExcludeMetadata = {}): void
    {
        // eslint-disable-next-line @typescript-eslint/ban-ts-comment
        // @ts-ignore
        const excludeMetadata: ExcludeMetadata = classTransformerDefaultMetadataStorage.findExcludeMetadata(target, propertyName);
        expect(excludeMetadata).toBeDefined();
        expect(excludeMetadata.options).toEqual(options);
    }

    public static TestTypeMetadata(target: Function, propertyName: string, type: Function): void
    {
        const typeMetadata: TypeMetadata = classTransformerDefaultMetadataStorage.findTypeMetadata(target, propertyName);
        expect(typeMetadata).toBeDefined();
        expect(typeMetadata.typeFunction()).toBe(type);
    }

    public static TestValidation(target: Function, propertyName: string, validationNames: ValidationTypes[]): void
    {
        const validationChecks: Array<Partial<ValidationMetadata>> = [];
        for (const validationName of validationNames)
        {
            validationChecks.push(expect.objectContaining({
                propertyName: propertyName,
                type: validationName
            }));
        }

        const validation: ValidationMetadata[] = getMetadataStorage().getTargetValidationMetadatas(target, ``, true, false).filter((value: ValidationMetadata) => value.propertyName === propertyName && value.type !== `customValidation`);
        expect(validation).toEqual(
            expect.arrayContaining(validationChecks)
        );
    }

    // eslint-disable-next-line @typescript-eslint/no-explicit-any
    public static TestConstraints(target: Function, propertyName: string, constraints: Array<{ name: string; options?: any[]; validationOptions?: ValidationOptions }>): void
    {
        const validationMetadatas: ValidationMetadata[] = getMetadataStorage().getTargetValidationMetadatas(target, ``, true, false)
            .filter((value: ValidationMetadata) => value.propertyName === propertyName && value.type === `customValidation`);

        // eslint-disable-next-line @typescript-eslint/no-explicit-any
        const constraintMetadatas: ConstraintMetadata[] = (getMetadataStorage() as any).constraintMetadatas
            .filter((constraint: ConstraintMetadata) => validationMetadatas.find((validation: ValidationMetadata) => validation.constraintCls === constraint.target));

        const constraintsCheck: Array<Partial<ConstraintMetadata>> = [];
        for (const constraint of constraints)
        {
            const constraintMetadata: ConstraintMetadata | undefined = constraintMetadatas.find((c: ConstraintMetadata) => c.name === constraint.name);
            expect(constraintMetadata).toBeDefined();

            const validationMetadata: ValidationMetadata | undefined = validationMetadatas.find((v: ValidationMetadata) => v.constraintCls === constraintMetadata?.target);
            expect(validationMetadata).toBeDefined();

            if (constraint.validationOptions)
            {
                expect(validationMetadata).toEqual(
                    expect.objectContaining(constraint.validationOptions)
                );
            }

            constraintsCheck.push(expect.objectContaining({name: constraint.name}));

            if (constraint.options)
            {
                expect(validationMetadata?.constraints.length).toBe(constraint.options.length);
                for (let i = 0; i < constraint.options.length; i++)
                {
                    expect(validationMetadata?.constraints[i]).toEqual(constraint.options[i]);
                }
            }
        }

        expect(constraintMetadatas).toEqual(
            expect.arrayContaining(constraintsCheck)
        );
    }
}

and here are test using it

    test(`paramOptions`, async() =>
    {
        ValidationConstraintUtils.TestExposeMetadata(target, `paramOptions`, {name: `param_options`});
        ValidationConstraintUtils.TestValidation(target, `paramOptions`, [ValidationTypes.CONDITIONAL_VALIDATION, ValidationTypes.NESTED_VALIDATION]);
    });

nolazybits avatar Mar 22 '21 01:03 nolazybits

acutally importing it as I shown above failed today. Another way was this import { defaultMetadataStorage as classTransformerDefaultMetadataStorage } from "class-transformer/cjs/storage"; with aliasing, otherwise it doesn't get picked... This is super weird...

TS 4.1.2 Node 14 Using ts-node latest

nolazybits avatar Mar 24 '21 03:03 nolazybits

Yeah, this is a tooling bug with the new format, I am kind of out of depth here, so I will ask about this on SO and hope someone knows whats up.

NoNameProvided avatar Mar 24 '21 08:03 NoNameProvided

@NoNameProvided is there any plan to make this API public so that other libraries can more easily and reliably interact with class-transformer? My use case is, I need to access user-defined type for nested objects on models, and the class-transformer is already storing/ requiring users to annotate this type, so I was hoping to reuse type metadata information that is already collected by the class-transformer. Also, Let me know if this is ever going to happen.

whimzyLive avatar Jul 03 '21 14:07 whimzyLive

Also voting to have defaultMetadataStorage as public API. It's very very useful to build advanced features on top of class-transformer. So please add support <3

kay-schecker avatar Sep 01 '21 21:09 kay-schecker

I would also greatly appreciate to have the metadata defaultMetadataStorage as public API.

DennisKuhn avatar Sep 10 '21 19:09 DennisKuhn

any update in here? we really need to use storage for implementing @nestjs/mapped-types in browser

xkguq007 avatar Sep 13 '21 11:09 xkguq007

I would also greatly appreciate having defaultMetadataStorage as a public API.

xerxes235 avatar Nov 03 '21 19:11 xerxes235

Any update on this? Without exposing the storage or at least exposing methods to retrieve the metadata based on a target any additional tooling won't be able to benefit from the metadata already generated by the decorators provided by class-transformer.

StefanSafeguard avatar Mar 04 '22 16:03 StefanSafeguard

I support making this public - we need access to it in order to get the metadata. Please make this public, guys. It's really not a big deal.

When I attempt to import {defaultMetadataStorage} from 'class-transformer/esm5/storage' I get

SyntaxError: Cannot use import statement outside a module

nosachamos avatar Dec 09 '22 21:12 nosachamos

Any updates? Would be good to prioritize this issue as it prevents class transformer from being used in new projects.

natejgardner avatar Jan 03 '23 06:01 natejgardner

Kindly make defaultMetadataStorage public.

aqibgatoo avatar Jan 18 '23 08:01 aqibgatoo

Importing it like this in my ts file worked for Me :)

// @ts-nocheck import { defaultMetadataStorage } from 'node_modules/class-transformer/esm5/storage.js';

class-transformer v. 0.5.1 TS 4.8.2 Node 18.12.1

krzysiek3d avatar Feb 08 '23 16:02 krzysiek3d

I also cannot load it, the import mentioned didn't work for me. Any luck making it public?

ptheofan avatar Mar 15 '23 23:03 ptheofan

In a deno project it worked for me only via commonjs import 🤷‍♂️

// @ts-ignore
import { defaultMetadataStorage } from "npm:[email protected]/cjs/storage.js";

zingmane avatar Mar 16 '23 15:03 zingmane

This works for me in version 0.14.0

import {getMetadataStorage} from 'class-validator';

While I've read the entire thread above. It wasn't clear the import problem had been fixed and this issue is still open.

Also, while I appreciate this is an internal feature of the library. It's not mentioned in the README file, and if you're like me going to Google or going to ChatGPT for a solution of "how do I get the metadata at run-time" then there is a lot of public information pointing to this function as a feature of the library.

It would be nice, if a footnote could be added to the README explaining how to import the method with a stern warning that it's not an official part of the API and subject to change. That would be good enough for most use cases.

codemile avatar May 29 '23 12:05 codemile

@codemile Thanks your discovery, but your solution is incompatible with class-validator-jsonschema, such as:

error TS2740: Type 'MetadataStorage' is missing the following properties from type 'MetadataStorage': _typeMetadatas, _transformMetadatas, _exposeMetadatas, _excludeMetadatas, and 20 more.

14             classTransformerMetadataStorage: getMetadataStorage(),
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/class-validator-jsonschema/build/options.d.ts:6:5
    6     classTransformerMetadataStorage?: ClassTransformerMetadataStorage;
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    The expected type comes from property 'classTransformerMetadataStorage' which is declared here on type 'Partial<IOptions>'

TechQuery avatar Jun 01 '23 17:06 TechQuery

Any chance metadata will be exported? My use case is to get an @Expose() metadata during serializing validation errors. Thnx.

ronskyi avatar Nov 24 '23 15:11 ronskyi

Hi , i'm not sure which is correct or not, i try :

import { MetadataStorage } from "class-transformer/types/MetadataStorage";

...
 const schemas = validationMetadatasToSchemas({
      // classTransformerMetadataStorage: defaultMetadataStorage,
      classTransformerMetadataStorage: new MetadataStorage(),
      refPointerPrefix: "#/components/schemas/",
    });

I read this and found it just new MetadataStorage() and export

taiwanhua avatar Jan 20 '24 18:01 taiwanhua