Docker compose persistence

[slawrence-miso]

When I try to use the docker compose examples I am only able to have persistence between container restarts. If I kill the containers using down I end up with errors such as this:

oracle-1     | ####################################################################
oracle-1     | CONTAINER: The following output is now from the alert_FREE.log file:
oracle-1     | ####################################################################
oracle-1     | ===========================================================
oracle-1     | 2025-01-22T15:04:24.567114+00:00
oracle-1     | Errors in file /opt/oracle/diag/rdbms/free/FREE/trace/FREE_p000_163.trc:
oracle-1     | ORA-01157: cannot identify/lock data file 24 - see DBWR trace file
oracle-1     | ORA-01110: data file 24: '/opt/oracle/product/23ai/dbhomeFree/dbs/DATA'
oracle-1     | 2025-01-22T15:04:24.568492+00:00
oracle-1     | Errors in file /opt/oracle/diag/rdbms/free/FREE/trace/FREE_ora_155.trc:
oracle-1     | ORA-01157: cannot identify/lock data file 24 - see DBWR trace file
oracle-1     | ORA-01110: data file 24: '/opt/oracle/product/23ai/dbhomeFree/dbs/DATA'
oracle-1     | Completed: ALTER DATABASE OPEN
oracle-1     | 2025-01-22T15:04:25.150297+00:00
oracle-1     | Errors in file /opt/oracle/diag/rdbms/free/FREE/trace/FREE_mz00_412.trc:
oracle-1     | ORA-01110: data file 24: '/opt/oracle/product/23ai/dbhomeFree/dbs/DATA'
oracle-1     | ORA-01565: Error identifying file /opt/oracle/product/23ai/dbhomeFree/dbs/DATA.
oracle-1     | ORA-27037: unable to obtain file status
oracle-1     | Linux-x86_64 Error: 2: No such file or directory
oracle-1     | Additional information: 7
oracle-1     | Checker run found 1 new persistent data failures

I am configuring the volume like this:

volumes:
  - oracle-volume:/opt/oracle/oradata

I found some discussions about using a DATAFILE option during the tablespace creation, and maybe that is an option that would work, but I was unable to find where that is happening.

[oraclesean]

The volume configuration you've defined creates an anonymous volume. Running compose down doesn't drop the volume, but it doesn't have a name that subsequent compose up commands recognize, so compose doesn't remount the volume. See https://docs.docker.com/reference/cli/docker/compose/down/ for more information.

To solve this, use a named volume bind-mount instead. I recommend this option, since it gives you control over the location of the database datafiles. Anonymous or unbound named volumes are saved under the protected /var/lib/docker directory (or its equivalent) where you have limited visibility. First, create a volume and assign it to a local directory:

docker volume create --opt type=none --opt o=bind --opt device=/some/directory oracledb_data

Then, reference the named volume in your compose specification:

services:
  oracledb:
<snip>
    volumes:
      - oracledb_data:/opt/oracle/oradata
<snip>

volumes:
  oracledb_data:
    external: true

If you prefer to use an unbound volume, the steps are:

docker volume create oracledb_data

The compose yaml file looks nearly identical; just exclude the external: true property within the volume definition:

services:
  oracledb:
<snip>
    volumes:
      - oracledb_data:/opt/oracle/oradata
<snip>

volumes:
  oracledb_data:

If you've run compose up and compose down, you'll likely have orphaned volumes that are taking up space and it's probably wise to clean them up. :) List them with:

docker volume ls

And remove them using:

docker volume rm <volume name>

(Running docker compose down -v removes volumes defined in the compose specification.)

HTH!

[slawrence-miso]

Ok, the first time I run this new volume configuration all seems good. I see my init.d script running, and I end up with a new folder in my current directory which is the host folder where I have pointed a new volume to.

The volume creation:

docker volume create --opt type=none --opt o=bind --opt device=/c/Users/MYUSER/Documents/dev/common/oracle/oradata

docker volume ls
local     oracledb_data

docker-compose config:

volumes:
      - type: bind
        source: /c/Users/MYUSER/Documents/dev/common/oracle/container-entrypoint-initdb.d
        target: /container-entrypoint-initdb.d
      - oracledb_data:/opt/oracle/oradata
 volumes:
  oracledb_data:
    external: true

The local oradata folder is populated:

$ find ./oradata/
./oradata/
./oradata/dbconfig
./oradata/dbconfig/FREE
./oradata/dbconfig/FREE/listener.ora
./oradata/dbconfig/FREE/orapwFREE
./oradata/dbconfig/FREE/spfileFREE.ora
./oradata/dbconfig/FREE/sqlnet.ora
./oradata/dbconfig/FREE/tnsnames.ora
./oradata/FREE
./oradata/FREE/control01.ctl
./oradata/FREE/control02.ctl
./oradata/FREE/FREEPDB1
./oradata/FREE/FREEPDB1/sysaux01.dbf
./oradata/FREE/FREEPDB1/system01.dbf
./oradata/FREE/FREEPDB1/temp01.dbf
./oradata/FREE/FREEPDB1/undotbs01.dbf
./oradata/FREE/FREEPDB1/users01.dbf
./oradata/FREE/DB_LOCAL
./oradata/FREE/DB_LOCAL/sysaux01.dbf
./oradata/FREE/DB_LOCAL/system01.dbf
./oradata/FREE/DB_LOCAL/temp01.dbf
./oradata/FREE/DB_LOCAL/undotbs01.dbf
./oradata/FREE/DB_LOCAL/users01.dbf
./oradata/FREE/pdbseed
./oradata/FREE/pdbseed/sysaux01.dbf
./oradata/FREE/pdbseed/system01.dbf
./oradata/FREE/pdbseed/temp01.dbf
./oradata/FREE/pdbseed/undotbs01.dbf
./oradata/FREE/redo01.log
./oradata/FREE/redo02.log
./oradata/FREE/redo03.log
./oradata/FREE/sysaux01.dbf
./oradata/FREE/system01.dbf
./oradata/FREE/temp01.dbf
./oradata/FREE/undotbs01.dbf
./oradata/FREE/users01.dbf

Hitting ctrl-c in docker-compose and re-run up and the DB starts back up ok.

After running docker-compose down and then up again I see this:

oracle-1  | CONTAINER: The following output is now from the alert_FREE.log file:
oracle-1  | ####################################################################
oracle-1  | DB_LOCAL(4):JIT: pid 161 requesting stop
oracle-1  | 2025-02-06T15:33:48.074736+00:00
oracle-1  | Errors in file /opt/oracle/diag/rdbms/free/FREE/trace/FREE_p000_161.trc:
oracle-1  | ORA-01157: cannot identify/lock data file 24 - see DBWR trace file
oracle-1  | ORA-01110: data file 24: '/opt/oracle/product/23ai/dbhomeFree/dbs/DATA'
oracle-1  | 2025-02-06T15:33:48.079663+00:00
oracle-1  | Errors in file /opt/oracle/diag/rdbms/free/FREE/trace/FREE_ora_153.trc:
oracle-1  | ORA-01157: cannot identify/lock data file 24 - see DBWR trace file
oracle-1  | ORA-01110: data file 24: '/opt/oracle/product/23ai/dbhomeFree/dbs/DATA'
oracle-1  | Completed: ALTER DATABASE OPEN
oracle-1  | 2025-02-06T15:33:50.249623+00:00
oracle-1  | Errors in file /opt/oracle/diag/rdbms/free/FREE/trace/FREE_mz00_410.trc:
oracle-1  | ORA-01110: data file 24: '/opt/oracle/product/23ai/dbhomeFree/dbs/DATA'
oracle-1  | ORA-01565: Error identifying file /opt/oracle/product/23ai/dbhomeFree/dbs/DATA.
oracle-1  | ORA-27037: unable to obtain file status
oracle-1  | Linux-x86_64 Error: 2: No such file or directory
oracle-1  | Additional information: 7
oracle-1  | Checker run found 1 new persistent data failures

So it persists between restarts but fails after down/up.

[oraclesean]

There's one subtle thing missing from the volume creation command:

docker volume create --opt type=none --opt o=bind \
       --opt device=/c/Users/MYUSER/Documents/dev/common/oracle/oradata

That creates a volume, but not a named volume. Try this:

docker volume create --opt type=none --opt o=bind \
       --opt device=/c/Users/MYUSER/Documents/dev/common/oracle/oradata oracledb_data

Since you're seeing files populated in the local device directory when starting compose, I think there's something else going on. docker volume ls is showing your named volume. To see what it's pointing at, run:

docker inspect oracledb_data

I did a quick test to see the difference.

First, I created an anonymous volume (without appending a name after the device spec):

# docker volume create --opt type=none --opt o=bind \
>        --opt device=/oradata/test0
c820584bc15662c8b2b5da0ced8a795311e3c58a71f9962c14d05007e7eb377c

Note that the output is a hash.

Next, I ran the command and added a volume name:

# docker volume create --opt type=none --opt o=bind
>        --opt device=/oradata/test1 named_volume
named_volume

Note the output returned the volume name.

I inspect the first volume:

# docker inspect c820584bc15662c8b2b5da0ced8a795311e3c58a71f9962c14d05007e7eb377c
[
    {
        "CreatedAt": "2025-02-06T15:54:11Z",
        "Driver": "local",
        "Labels": {
            "com.docker.volume.anonymous": ""
        },
        "Mountpoint": "/docker/.docker/volumes/c820584bc15662c8b2b5da0ced8a795311e3c58a71f9962c14d05007e7eb377c/_data",
        "Name": "c820584bc15662c8b2b5da0ced8a795311e3c58a71f9962c14d05007e7eb377c",
        "Options": {
            "device": "/oradata/test0",
            "o": "bind",
            "type": "none"
        },
        "Scope": "local"
    }
]

Then the second:

# docker inspect named_volume
[
    {
        "CreatedAt": "2025-02-06T15:54:24Z",
        "Driver": "local",
        "Labels": null,
        "Mountpoint": "/docker/.docker/volumes/named_volume/_data",
        "Name": "named_volume",
        "Options": {
            "device": "/oradata/test1",
            "o": "bind",
            "type": "none"
        },
        "Scope": "local"
    }
]

Note the differences (unnamed vs. named):

        "Labels": { "com.docker.volume.anonymous": "" },
        "Labels": null,

        "Mountpoint": "/docker/.docker/volumes/c820584bc15662c8b2b5da0ced8a795311e3c58a71f9962c14d05007e7eb377c/_data",
        "Mountpoint": "/docker/.docker/volumes/named_volume/_data",

        "Name": "c820584bc15662c8b2b5da0ced8a795311e3c58a71f9962c14d05007e7eb377c",
        "Name": "named_volume",```

What I think is happening here is that the anonymous volume was bound to the same location as the named volume, which is completely legal:

# docker volume create --opt type=none --opt o=bind  --opt device=/oradata/test1
a1a0b53d61a4b5bf3ae353d22fb186edc786a3185ea7bd4214e389b76e217caa

# docker inspect a1a0b53d61a4b5bf3ae353d22fb186edc786a3185ea7bd4214e389b76e217caa
[
    {
        "CreatedAt": "2025-02-06T16:15:33Z",
        "Driver": "local",
        "Labels": {
            "com.docker.volume.anonymous": ""
        },
        "Mountpoint": "/docker/.docker/volumes/a1a0b53d61a4b5bf3ae353d22fb186edc786a3185ea7bd4214e389b76e217caa/_data",
        "Name": "a1a0b53d61a4b5bf3ae353d22fb186edc786a3185ea7bd4214e389b76e217caa",
        "Options": {
            "device": "/oradata/test1",
            "o": "bind",
            "type": "none"
        },
        "Scope": "local"
    }
]

compose up references the named volume. When you run compose down I'm guessing it somehow recognizes the filesystem as also being bound to an anonymous volume and removes it. To confirm this, can you:

• Run docker volume ls and docker inspect oracledb_data.
• Run docker ps -a to be sure there are no other running containers.
• Run the original volume create command, without the name.
• Re-run docker volume ls and see if there's a new, anonymous volume. It would be worthwhile to run inspect on both volumes, too.
• Run compose up as before.
• After the DB comes up, confirm that the local directory is populated.
• Check the mounts and bindings: docker inspect --format='{{ range .Mounts }} {{printf "%-30s %-80s %-80s\n" .Name .Destination .Source }} {{ end }}' <container name>
• Run compose down.
• Re-check the output of docker volume ls, which I believe will now show only one volume.
• Check the output of docker volume ls, which I expect to show only the original named volume.
• Check the contents of the local directory and see if files were removed. Again, that's my best guess of what's happening.

Or, you could just drop any existing volumes, create a new, named volume, and try compose up and compose down to see whether things persist.

[slawrence-miso]

Ok, I made sure to use the named version of the volume command.

This data is persisting technically, and I can see files in the folder after stopping and starting the docker-compose service but oracle is complaining on the second up run the same way as my comment from earlier:

oracle │ #################################################################### oracle │ CONTAINER: The following output is now from the alert_FREE.log file: oracle │ #################################################################### oracle │ DB_LOCAL(4):JIT: pid 163 requesting stop oracle │ 2025-02-06T19:14:35.985880+00:00 oracle │ Errors in file /opt/oracle/diag/rdbms/free/FREE/trace/FREE_p000_163.trc: oracle │ ORA-01157: cannot identify/lock data file 24 - see DBWR trace file oracle │ ORA-01110: data file 24: '/opt/oracle/product/23ai/dbhomeFree/dbs/DATA' oracle │ 2025-02-06T19:14:35.988699+00:00 oracle │ Errors in file /opt/oracle/diag/rdbms/free/FREE/trace/FREE_ora_155.trc: oracle │ ORA-01157: cannot identify/lock data file 24 - see DBWR trace file oracle │ ORA-01110: data file 24: '/opt/oracle/product/23ai/dbhomeFree/dbs/DATA' oracle │ Completed: ALTER DATABASE OPEN oracle │ 2025-02-06T19:14:37.117857+00:00 oracle │ Errors in file /opt/oracle/diag/rdbms/free/FREE/trace/FREE_mz00_412.trc: oracle │ ORA-01110: data file 24: '/opt/oracle/product/23ai/dbhomeFree/dbs/DATA' oracle │ ORA-01565: Error identifying file /opt/oracle/product/23ai/dbhomeFree/dbs/DATA. oracle │ ORA-27037: unable to obtain file status oracle │ Linux-x86_64 Error: 2: No such file or directory oracle │ Additional information: 7 oracle │ Checker run found 1 new persistent data failures

[oraclesean]

Hmm. OK, let me take a look and see if I can duplicate this locally (hopefully over the weekend). Do you mind sharing your compose.yaml file?

[oraclesean]

Sorry, also the output of docker info and the version of compose you're running. Thanks!

[slawrence-miso]

Looks related to this:

https://github.com/oracle/docker-images/blob/main/OracleDatabase/SingleInstance/FAQ.md#ora-01157-cannot-identifylock-data-file

How/where do I set that path?

[slawrence-miso]

I think this might be caused by my CREATE TABLESPACE DATA DATAFILE I am running in my initdb.d scripts. Trying to use the fill filepath for the datafile.