vitejs
Built-in Support for Generating Target Modules from Source Modules
Description
In many development scenarios, we need to generate target modules dynamically based on source modules. Currently, this requires custom plugins or scripts, which can be cumbersome to manage and integrate.
Anyway, integrating compile-time code generation capabilities into vite seems natural to me. I tried searching but found no related issues. Maybe the keywords I used were incorrect.
Suggested solution
Here is a sample implementation using a custom Vite plugin:
Capabilities:
Recursively Traverse Source Directory
The plugin can recursively traverse the specified source directory (/src) to find all files, including those in subdirectories. This is achieved using the fs.readdirSync and fs.statSync functions for recursive directory traversal.
Identify and Process Source Modules with Specific Suffix
The plugin identifies source module files with a specific suffix (e.g., .source.js) and uses these files as the basis for generating target module files.
Generate Target Module Files
For each identified source module file, the plugin generates a corresponding target module file. The generated target module file uses the same base name as the source module file but with a different suffix (e.g., .gen.js).
Generate Content Hash
The plugin generates a content hash for each source module file using the crypto.createHash function. This hash is included in the target module file to track changes in the file content.
Add Comments Indicating Source Module Path
At the beginning of the generated target module file, the plugin adds a comment indicating the source module file from which it was generated and includes the content hash.
Handle Hot Updates
The plugin supports hot module replacement. When a source module file changes, the plugin automatically regenerates the corresponding target module file and notifies the Vite development server to reload the page using the server.ws.send function.
Log Generation Process
The plugin logs information to the console during the generation and regeneration of target module files, providing clear feedback on which files have been processed.
Code:
import fs from 'fs-extra';
import path from 'path';
import crypto from 'crypto';
function generateHash(content) {
return crypto.createHash('md5').update(content).digest('hex');
}
function getAllFiles(dirPath, arrayOfFiles) {
const files = fs.readdirSync(dirPath);
arrayOfFiles = arrayOfFiles || [];
files.forEach((file) => {
if (fs.statSync(path.join(dirPath, file)).isDirectory()) {
arrayOfFiles = getAllFiles(path.join(dirPath, file), arrayOfFiles);
} else {
arrayOfFiles.push(path.join(dirPath, file));
}
});
return arrayOfFiles;
}
async function executeModule(modulePath) {
const { generateContent } = await import(modulePath);
return generateContent();
}
function generateModulesPlugin() {
return {
name: 'generate-modules',
buildStart() {
const sourceDir = path.resolve(__dirname, '../../src');
const files = getAllFiles(sourceDir);
files.forEach(async (file) => {
if (file.endsWith('.source.js')) {
const originalPath = path.resolve(file);
const generatedPath = originalPath.replace('.source.js', '.gen.js');
const content = fs.readFileSync(originalPath, 'utf-8');
const hash = generateHash(content);
const moduleContent = await executeModule(originalPath);
const finalContent = `
// Generated from ${originalPath} with hash ${hash}
${moduleContent}
`;
fs.outputFileSync(generatedPath, finalContent);
console.log(`Generated ${generatedPath} from ${originalPath}`);
}
});
},
handleHotUpdate({ file, server }) {
if (file.endsWith('.source.js')) {
const originalPath = path.resolve(file);
const generatedPath = originalPath.replace('.source.js', '.gen.js');
const content = fs.readFileSync(originalPath, 'utf-8');
const hash = generateHash(content);
executeModule(originalPath).then((moduleContent) => {
const finalContent = `
// Generated from ${originalPath} with hash ${hash}
${moduleContent}
`;
fs.outputFileSync(generatedPath, finalContent);
console.log(`Regenerated ${generatedPath} from ${originalPath} due to changes`);
server.ws.send({
type: 'full-reload',
path: '*'
});
});
}
}
};
}
export default generateModulesPlugin;
(The demo code is generated using GPT and may still be inaccurate.)
Alternative
Use configuration files to agree on the activation of source modules and the correspondence between the file names of source modules and the target modules they generate, to avoid naming conflicts that may be caused by .source.js or gen.js suffixes.
Additional context
No response
Validations
- [X] Follow our Code of Conduct
- [X] Read the Contributing Guidelines.
- [X] Read the docs.
- [X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.
I don't quite understand the feature request:
- What are you generating from the source file?
- Are you using a tool or compiler to generate it?
- What about using a Vite plugin with the
transform()hook to transform the code so you don't need a separate file? - What kind of API are you thinking that Vite could expose?
Otherwise this seems fine to be implemented externally as a Vite plugin.
I have indeed written an external plugin to meet the needs of dynamically generating files. This is very simple, but the extended capabilities are exciting to me (very cost-effective).
I currently use "source file" to dynamically generate a list of files of a specified file type in a specified directory (for example, /src/lib/*_storage_io.js). (Configure vite.config.js to call a js program at each compilation, this js program will scan the /src/lib/ directory, and then create a new file /src/storage_list.json to output the scanned information to it)
If you consider the potential use, this may encourage another style of programming paradigm. A simple example: suppose I want to design landing pages for 1,000 products. Using the method here, I will be able to hard-code the url of each product's image resource and dynamically generate 1,000 html without any js code. Is this very different from pages that must be implemented using javascript?
Assuming that Vite has this feature built in, I expect it to globally recognize specific file suffixes in the project, such as .source.js, and then automatically execute the code in it at compile time. (Or consider providing another suffix: .mustache.js so that during compilation, the code in it can be directly replaced with a template to generate a .js with the same name in the same path)
Compared to the transform() method, this will do more things in the compiler, and the generated target code will be more concise and diverse.
If Vite has this method built in, I think it would be very cool. Of course, it is also easy for users to implement it themselves using external plug-ins.
If I understand correctly, you want an easy way to "for these source files, compile with this js program, put them in that directory"? Vite's and Rollup's transform flow intentionally pushes against generating files like this, since it can be achieved with virtual modules.
I understand though that generating the files explicitly in the fs has its benefits too, but at the moment, I don't think it's something Vite would support first-class. However, it may be easier to test your idea with the community now if you can share a Vite plugin that implements this feature generically. I'll close this for now.