vagrant-env Plugin Issue

[PaulNeumann]

Many of the projects in this repository can optionally use the vagrant-env plugin to make configuration easier. However, in Vagrant 2.4.3 and above, the plugin doesn't work correctly. For any project that loads the plugin, vagrant commands fail with an error message similar to the following:

Vagrant failed to initialize at a very early stage:

There was an error loading a Vagrantfile. The file being loaded
and the error message are shown below. This is usually caused by
an invalid or undefined variable.

Path: /home/username/.vagrant.d/gems/3.3.6/gems/dotenv-0.11.1/lib/dotenv.rb
Line number: 0
Message: undefined method `exists?'

This happens because the vagrant-env plugin depends on a very old version of the dotenv gem. The gem uses the File.exists? method, which was removed in Ruby 3.2.0. Vagrant 2.4.3 and above bundle Ruby 3.3.x, which doesn't include the method. (See https://github.com/hashicorp/vagrant/issues/13550.)

Fortunately, it's easy to patch the dotenv gem to make the plugin work correctly.

For Linux hosts (also works in Git Bash on Windows hosts):

find ~/.vagrant.d/gems/*/gems/dotenv-0.11.1/lib -maxdepth 1 -name 'dotenv.rb' \
  -type f -exec sed -i -e 's/File.exists?/File.exist?/g' {} \;

For Windows hosts (works with both Windows PowerShell 5.1 and PowerShell 7):

(Get-Item -Path "~\.vagrant.d\gems\*\gems\dotenv-0.11.1\lib\dotenv.rb").ForEach( {
    (Get-Content -Path $PSItem -Raw).Replace('File.exists?', 'File.exist?') | `
        Set-Content -Path $PSItem
} )

The vagrant-env plugin hasn't been updated since December 2015, so it seems unlikely that it will be modified to use a newer version of the gem.

I don't know the best way to address this, and it's not my decision. Possibilities include:

1. Do nothing. The problem hasn't been reported as an issue in this repository, so it may not be affecting many people (or those affected have implemented the workaround).

2. Document the workaround in the top-level README.

3. Use a different plugin with similar functionality. For example, the vagrant-readenv plugin is a fork of vagrant-env that uses a newer version of the dotenv gem.

   (I've tested the vagrant-readenv plugin, and it seems to work well. It requires only one code change in the Vagrantfile (if Vagrant.has_plugin?("vagrant-readenv")).)

4. Switch to a different configuration method.

What are your thoughts?

[AmedeeBulle]

People might still use older versions

We probably should move to something well maintained

[PaulNeumann]

That makes sense. I'll try to come up with a proposal for discussion.

[PaulNeumann]

@AmedeeBulle YAML seems to be a popular method for configuring Vagrant builds. Some of the projects in this repository use it. It's well-maintained, and it doesn't require a plugin.

I'd like to preserve the ability to configure the build using environment variables, with a committed .env-type file, and an optional .env.local-type file that isn't committed. That functionality is very useful.

I think the following could work. It requires only minimal code changes.

First, convert the .env file to config.yml. For example, for the OracleDatabase/23ai-Free project:

---
# comments removed for clarity, but would be present in the "real" file
VM_NAME: 'oracle23ai-free-vagrant'
VM_MEMORY: 2300
VM_SYSTEM_TIMEZONE: ''
VM_DB_VERSION: 'latest'
VM_KEEP_DB_INSTALLER: false
VM_ORACLE_CHARACTERSET: 'AL32UTF8'
VM_LISTENER_PORT: 1521
VM_ORACLE_PWD: ''

Then, in the Vagrantfile, the only changes needed would be adding require 'yaml' and replacing the code that currently looks like:

  # Use vagrant-env plugin if available
  if Vagrant.has_plugin?("vagrant-env")
    config.env.load(".env.local", ".env") # enable the plugin
  end

with the following:

  # Load configuration parameters into environment
  ['config.yml.local', 'config.yml'].each do |f|
    cfg = File.file?(f) ? YAML.safe_load_file(f, freeze: true) : {}
    cfg && cfg.each do |key, value|
      key = key.upcase
      (!ENV[key] || ENV[key].empty?) && ENV[key] = value.to_s
    end
  end

This does the same thing that the vagrant-env plugin currently does. It maintains the same precedence (environment variables, values in config.yml.local, values in config.yml, Vagrantfile definitions). I've tested it, and it seems to work as intended.

What do you think?

[AmedeeBulle]

@PaulNeumann Thanks for researching. I like the idea, in particular the fact that it is supported natively (no plugin required).

From a cosmetic standpoint, I would name the files config.yaml and config.local.yaml, which is more standard.

@gvenzl / @scoter-oracle -- as codeowners, are you OK with this change?

[PaulNeumann]

@AmedeeBulle Thank you. If this moves forward, I'll use the filenames you recommended.

[gvenzl]

I'm OK with this approach!

[PaulNeumann]

@gvenzl Thanks! I'll work on these changes for the single-instance database projects and submit a PR.

[PaulNeumann]

The single-instance database projects have been updated to optionally use YAML, rather than the vagrant-env plugin. There are 7 other projects that currently can use the plugin, if it's installed:

• ContainerRegistry
• OCNE
• OracleGoldenGate/12.2.0.1
• OracleGoldenGate/19.3.0
• OracleLinux/7
• OracleLinux/8
• OracleLinux/9

The Ocr-Yum-Mirror project also refers to the vagrant-env plugin in both README.md and the Vagrantfile, but no .env file is included. I don't think the plugin is actually used for configuration, so I think the references to the plugin could be removed from both README.md and the Vagrantfile.

[PaulNeumann]

@AmedeeBulle Now that the Oracle Linux 10 box is available (thank you!), I built a Vagrant project with it, based on the OracleLinux/9 project. I used the code above to configure the build using YAML files, and I ran into an issue.

The .env files for the OL 7/8/9 projects include lines like EXTEND=$EXTEND,container-tools and EXPOSE=$EXPOSE,8080:80. The vagrant-env plugin (really, the dotenv gem) expands variables like $EXTEND. The code above doesn't, so an error message (e.g., "Extension $EXTEND does not exist -- ignored") is displayed when the Vagrantfile runs. The error doesn't stop the build, and any EXTEND or EXPOSE definitions work as intended.

The variable expansion code is in the file ~/.vagrant.d/gems/3.3.8/gems/dotenv-0.11.1/lib/dotenv/substitutions/variable.rb for Vagrant 2.4.5+. Something similar could be added to the YAML configuration code, but I'm not sure that would make sense for this use case. EXTEND=$EXTEND,extension1,extension2 is useful if one wants to source a .env file, but here, I don't think we want to modify previously-defined environment variables. So, I wonder whether it would be better to remove $EXTEND and $EXPOSE from the configuration files.

I could be missing something obvious, though. What do you think?

[AmedeeBulle]

@PaulNeumann -- good catch!

It has been quite a wile, I had to look back at the code 😉

The idea of all this was to be able to easily extend the Vagrantfile without modifying anything else and not having to know something about other extensions. By playing with the variables, you could say: add this extension to the existing list and add these port mappings to the mapping list.

But it is more a proof of concept / demo than anything else, I don't think anybody is actually using this to that extend. We can simplify in having a single entry for each parameter rather than building them incrementally:

EXTEND=<list of all extensions>
EXPOSE=<list of all exposed ports>

which would be equivalent

[PaulNeumann]

@AmedeeBulle OK. Thanks!

If you'd like, I could submit a PR for an OL 10 project for consideration. I could also update the OL 7/8/9 projects, if that would help. I don't think the code owner has approved the change from the plugin to YAML for configuration, though.