Add .schema files to openldap

[mathieudevos]

Hey,

How can you rerun the base ldap.conf so that new .schema files can be added to the setup?

I have .schema files to emulate a windows AD, but I cannot reconfigure the whole thing since all the base files are deleted.

Any pointers on how to get this running would be great.

[obourdon]

@mathieudevos not sure I fully understand what you are asking here. Do you already have a running instance of osixia/openldap for which you want to change configuration and add .schema files ? Or do you simply want to add custom .schema files to a non-existing osixia/openldap instance ? Thanks for clarifying

[mathieudevos]

Hey,

Thanks for the quick response. I have a bunch of .schema files from here: https://github.com/dkoudela/active-directory-to-openldap/tree/master/schema

And they need to be in the following order for slaptest to compile:

microsoftattributetype
microsoftattributetypestd
core (modified core from that url)
... everything else
microsoftobjectclass

To do that I will need to most likely modify the boostrap script. However I'm at a loss here with doing that. Currently I rebuilt the container with a hard copy of that folder, manually edited the schema-to-ldif.sh file to do the following:

echo "include /container/service/slapd/schemas/microsoftattributetype.schema" >> convert.dat
echo "include /container/service/slapd/schemas/microsoftattributetypestd.schema" >> convert.dat
echo "include /container/service/slapd/schemas/core.schema" >> convert.dat
echo "include /container/service/slapd/schemas/cosine.schema" >> convert.dat
echo "include /container/service/slapd/schemas/inetorgperson.schema" >> convert.dat

if [ -e "/etc/ldap/schema/rfc2307bis.schema" ]; then
  echo "include /etc/ldap/schema/rfc2307bis.schema" >> convert.dat
else
  echo "include /etc/ldap/schema/nis.schema" >> convert.dat
fi


for schema in ${SCHEMAS} ; do
    echo "include ${schema}" >> convert.dat
done

echo "include /container/service/slapd/schemas/microsoftobjectclass.schema" >> convert.dat

Which passes the slaptest - however the server remains on "slapd starting" and never progresses beyond that. Going into the container and going to the /etc/ldap/slapd.d/cn=schema folder shows that there's just 5 files:

cn={0}core.ldif 
cn={1}cosine.ldif 
cn={2}nis.ldif 
cn={3}inetorgperson.ldif 
cn={4}ppolicy.ldif

Which has the microsoft ones missing from it.

Other ways around this? Or how would I just start with something like this? More than happy to contribute to this so that we can have a flag for windows openldap settings if we figure this out.

Fast edit: to answer your question, I will happily start from scratch on this, so a non-existing one would most likely be the valid format. I have a bunch of groups & users from a windows AD, and would like to duplicate them in an openldap container so I can test-run my program to connect to it :-).

[dylanpiergies]

I'm trying to do the same thing. The repository that @mathieudevos mentions above contains a customised schema that more closely matches that of Active Directory, along with a slapd.conf that contains, amongst one or two other things, some rewrite rules that influence the search behaviour. Any guidance on how we can apply this would be appreciated.

[dylanpiergies]

@obourdon To clarify, my intention here is to bootstrap a fresh container with the modified schema and configuration in a new image derived from osixia/openldap.

[fbartels]

And they need to be in the following order for slaptest to compile:

have you tried naming your files like 01microsoftattributetype.schema, 02microsoftattributetypestd.schema ... ?

[quanah]

There are two different things at play here.

".schema" files are traditionally only for slapd.conf ".ldif" files are for use with cn=config

Second, I would note that OpenLDAP master now has an AD schema (msuser.schema/msuser.ldif).

Generally one should not be in the business of converting schema, since OpenLDAP already ships with .ldif format schema files for use with cn=config.

[guimaluf]

I'm struggling here to do some similar thing. Add custom schemas to this openldap setup

[Guarionex]

I'm struggling as well. Trying to add AD-LDS schema to OpenLdap to no success.

[monclaf]

Hey, I would like to duplicate data from an existing Ldap server (OpenDJ) to an osixia/openLap container. I've exported data as an ldif file and also schema as another ldif file with LDap Admin tool. Default osixia schema is different from the one I've exported, so I would like to load at container creation the schema I would like. I've tried to do like the "Seed ldap database with ldif" paragraph without success. Does someone get succeed in defining its own schema ? Thanks for any help.

[clayrisser]

I'm wondering the same thing. How do get this to use active directory schema?

https://github.com/dkoudela/active-directory-to-openldap/tree/master/schema

@mathieudevos any luck?

[clayrisser]

There are two different things at play here.

".schema" files are traditionally only for slapd.conf ".ldif" files are for use with cn=config

Second, I would note that OpenLDAP master now has an AD schema (msuser.schema/msuser.ldif).

Generally one should not be in the business of converting schema, since OpenLDAP already ships with .ldif format schema files for use with cn=config.

How would I used msuser? I cannot find it in the docker image.

@quanah

[lvillaca]

I also had this issue, and tried editing schema-to-ldif.sh with no luck. I was further able to add it by producing a new Docker image, copying my new schema (via Dockerfile) to "/container/service/slapd/assets/config/bootstrap/schema/" - see custom schema

[Kramins]

I am also trying to figure out a solution to adding a new schema to an existing container. I am trying to add postfix-book.schema I managed to convert it to an ldif by creating a conf like so

include /etc/ldap/schema/core.schema
include /etc/ldap/schema/cosine.schema
include /etc/ldap/schema/inetorgperson.schema
include /ldif/postfix-book.schema

but I am unsure what to do with the generated files and directory, truthfully I was expecting it to generate one file.

can I just add the schema file to /container/service/slapd/assets/config/bootstrap/ldif/custom/

[mckenziec]

Has anyone made progress on this topic? I'm also in the midst of trying to incorporate an ActiveDirectory lite schema into osixia and having no end of troubles.

[quanah]

You might find this useful. It's a general guide on how to convert .schema files to .ldif files for use with cn=config.

[mckenziec]

Thanks. I was writing a reply about the dependency hell I couldn't resolve, but I actually managed to convert the schema files included in https://github.com/dkoudela/active-directory-to-openldap to ldif.

Here's the config I used. The order of the schema files is key:

include microsoftattributetypestd.schema  
include microsoftattributetype.schema     
include core.schema  
include cosine.schema    
include inetorgperson.schema
include nis.schema
include microsoftobjectclass.schema      
include collective.schema
include dyngroup.schema
include misc.schema
include corba.schema       
include java.schema                       
include openldap.schema
include pmi.schema
include ppolicy.schema
include duaconf.schema

I'm at the same point as Karmis. What do I do with the resulting directory ldif files? Do I just mount the direct as a volume in osixia? Thanks.

[mckenziec]

Also as quanah suggested, I grabbed the msusers.ldif and tried messing around it.

Simply adding it as a custom ldif didn't work. There are complaints about a duplicate attribute. BTW, the encoded characters are different everytime I start the osixia container. I make sure there's no persistent storage of the ldap data.

Add custom bootstrap ldif...
Processing file /container/run/service/slapd/assets/config/bootstrap/ldif/custom/msuser.ldif
5ef26d9a conn=1024 fd=12 ACCEPT from PATH=/var/run/slapd/ldapi (PATH=/var/run/slapd/ldapi)
5ef26d9a conn=1024 op=0 BIND dn="" method=163
5ef26d9a conn=1024 op=0 BIND authcid="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" authzid="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth"
5ef26d9a conn=1024 op=0 BIND dn="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" mech=EXTERNAL sasl_ssf=0 ssf=71
5ef26d9a conn=1024 op=0 RESULT tag=97 err=0 text=
5ef26d9a conn=1024 op=1 ADD dn="cn=msuser,cn=schema,cn=config"
5ef26d9a conn=1024 op=1 RESULT tag=105 err=80 text=olcAttributeTypes: Duplicate attributeType: "�~4v"
5ef26d9a conn=1024 op=2 UNBIND
5ef26d9a conn=1024 fd=12 closed
5ef26d9a conn=1025 fd=12 ACCEPT from IP=127.0.0.1:58360 (IP=0.0.0.0:389)
5ef26d9a conn=1025 op=0 BIND dn="cn=admin,dc=csdpac,dc=demo" method=128
5ef26d9a conn=1025 op=0 BIND dn="cn=admin,dc=csdpac,dc=demo" mech=SIMPLE ssf=0
5ef26d9a conn=1025 op=0 RESULT tag=97 err=0 text=
5ef26d9a conn=1025 op=1 ADD dn="cn=msuser,cn=schema,cn=config"
5ef26d9a conn=1025 op=1 RESULT tag=105 err=50 text=
5ef26d9a conn=1025 op=2 UNBIND
5ef26d9a conn=1025 fd=12 closed
ldap_add: Other (e.g., implementation specific) error (80)
	additional info: olcAttributeTypes: Duplicate attributeType: "�~4v"
adding new entry "cn=msuser,cn=schema,cn=config"

ldap_add: Insufficient access (50)
adding new entry "cn=msuser,cn=schema,cn=config"
*** /container/run/startup/slapd failed with status 50

[mckenziec]

The admin account doesn't have access to cn=config. I can't view the schema of any running osixia container. If I embed msuser.lidf as /container/service/slapd/assets/config/bootstrap/ldif/06-msuser.ldif in a Docker image that's FROM the osixia image, I still get:

adding new entry "cn=msuser,cn=schema,cn=config"
ldap_add: Insufficient access (50)

Maybe I'm going crazy, but I thought seeding the osixia container would allow for schema level seeding. What am I missing? I'm on the verge of throwing the whole thing out.

[quanah]

There seems to be something seriously wrong with the msuser LDIF file you're adding, given that it prints out those characters. I would note that OpenLDAP already has this file converted for use so you shouldn't need to convert it yourself.

[mckenziec]

Hi quanah.

That's the ldif I'm using.

curl  https://git.openldap.org/openldap/openldap/-/raw/master/servers/slapd/schema/msuser.ldif -o ./ldif/msuser.ldif

Here's my Dockerfile:

FROM osixia/openldap
COPY ${PWD}/ldif/msuser.ldif /container/service/slapd/assets/config/bootstrap/ldif/06-msuser.ldif

I build it tagged as "test/test-ldap", and run with compose:

  ldap:
    image: test/test-ldap:latest
    container_name: ldap
    domainname: "test.local"
    hostname: "ldap"
    command: --copy-service --loglevel debug
    ports:
      - "389:389"
    env_file:
      - ${PWD}/ldap.env

My ldap.env just has this in it:

LDAP_LOG_LEVEL=256
LDAP_ORGANISATION=Test Inc.
LDAP_DOMAIN=test.local
LDAP_BASE_DN=dc=test,dc=local
LDAP_ADMIN_PASSWORD=admin
LDAP_TLS=false

Docker logs for the container indicates the same problem, no matter how I approach it:

...
5ef29060 conn=1024 fd=12 ACCEPT from PATH=/var/run/slapd/ldapi (PATH=/var/run/slapd/ldapi)
5ef29060 conn=1024 op=0 BIND dn="" method=163
5ef29060 conn=1024 op=0 BIND authcid="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" authzid="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth"
5ef29060 conn=1024 op=0 BIND dn="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" mech=EXTERNAL sasl_ssf=0 ssf=71
5ef29060 conn=1024 op=0 RESULT tag=97 err=0 text=
5ef29060 conn=1024 op=1 ADD dn="cn=msuser,cn=schema,cn=config"
5ef29060 conn=1024 op=1 RESULT tag=105 err=80 text=olcAttributeTypes: Duplicate attributeType: "�~4�"
5ef29060 conn=1024 op=2 UNBIND
5ef29060 conn=1024 fd=12 closed
5ef29060 conn=1025 fd=12 ACCEPT from IP=127.0.0.1:34038 (IP=0.0.0.0:389)
5ef29060 conn=1025 op=0 BIND dn="cn=admin,dc=test,dc=local" method=128
5ef29060 conn=1025 op=0 BIND dn="cn=admin,dc=test,dc=local" mech=SIMPLE ssf=0
5ef29060 conn=1025 op=0 RESULT tag=97 err=0 text=
5ef29060 conn=1025 op=1 ADD dn="cn=msuser,cn=schema,cn=config"
5ef29060 conn=1025 op=1 RESULT tag=105 err=50 text=
5ef29060 conn=1025 op=2 UNBIND
5ef29060 conn=1025 fd=12 closed
ldap_add: Other (e.g., implementation specific) error (80)
	additional info: olcAttributeTypes: Duplicate attributeType: "�~4�"
adding new entry "cn=msuser,cn=schema,cn=config"

ldap_add: Insufficient access (50)
adding new entry "cn=msuser,cn=schema,cn=config"
*** /container/run/startup/slapd failed with status 50

*** Run commands before finish...
*** Killing all processes...
5ef29060 daemon: shutdown requested and initiated.
5ef29060 slapd shutdown: waiting for 0 operations/tasks to finish
5ef29060 slapd stopped.

I've tried making is a custom volume mount, embedding it in a different location and setting LDAP_SEED_INTERNAL_LDIF_PATH, and as you can see above, I embedded it in my own Docker image in the same ldif directory that the osixia container will run last.

The default ldif files are run, including setting the admin password. It's when the msuser.ldif is processed that things fall apart. It looks like cn=admin doesn't have access to cn=config.

I don't doubt there could be something wrong with the ldif, or my apporach, but out-of-the-box I can access/view the installed default schema. The admin account doesn't seem to have the ACLs needed to update things.

I have no clue where those attributeType characters are coming from.

[mckenziec]

I'm not familiar enough with the ldif ACL syntax to say if the included one would impact the msuser.ldif:

dn: olcDatabase={1}{{ LDAP_BACKEND }},cn=config
changetype: modify
delete: olcAccess
-
add: olcAccess
olcAccess: to * by dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth manage by * break
olcAccess: to attrs=userPassword,shadowLastChange by self write by dn="cn=admin,{{ LDAP_BASE_DN }}" write by anonymous auth by * none
olcAccess: to * by self read by dn="cn=admin,{{ LDAP_BASE_DN }}" write by * none

https://github.com/osixia/docker-openldap/blob/stable/image/service/slapd/assets/config/bootstrap/ldif/02-security.ldif

There was a commit to it in April: https://github.com/osixia/docker-openldap/issues/402

[quanah]

I'm not familiar enough with the ldif ACL syntax to say if the included one would impact the msuser.ldif:

dn: olcDatabase={1}{{ LDAP_BACKEND }},cn=config

The Osixia container already provides full write access to the cn=config db for the root user over ldapi. There is nothing you need to change in regards to the ACLs. The above change also is completely invalid since it's modifying the olcDatabase={1}mdb,cn=config database access rules and not the access rules for the cn=config DB.

This particular error is bizarre, as it seems to indicate that somehow you have gotten non-ascii text in your LDIF file:

Duplicate attributeType: "�~�4�"

I would note there is a typo in your curl command, you have msuser.lidf instead of msuser.ldif as the destination object. Is it possible your install is picking up something other than the actual msuser.ldif from the OpenLDAP repo?

[mckenziec]

Thanks quanah. I won't pretend to understand openldap enough to diagnose this. That is a typo in my post, not in how I downloaded the file. Sorry about that.

[mckenziec]

I don't know where the (maybe utf-8) characters are coming from. It's literally the 1st non-comment entry in the msuser.ldif. I grepped through all the olcAttributeTypes and they aren't complicated. I downloaded the file directly to my CentOS8 dev env, then copy it into the image from my Dockerfile.

I also blew away all my osixia images and rebuilt. Then I inspected the msuser.ldif file inside the container and it looks the same.

[752319951]

Hello, what's the correct way to add a custom .schema file based on the above discussion？

[Kumassy]

/service/slapd/assets/schema-to-ldif.sh converts .schema files into .ldif, so you don't need to convert .schema files manually:

https://github.com/osixia/docker-openldap/blob/a3f7fb31ed882180bff6cd9fdd3182f2c81ea612/image/service/slapd/assets/schema-to-ldif.sh

[Kumassy]

@752319951 Only you have to do is

• Mount .schema files to /container/service/slapd/assets/config/bootstrap/schema/
• Start container with --copy-service (See: https://github.com/osixia/docker-openldap/issues/301#issuecomment-724669148, https://github.com/osixia/docker-openldap#seed-ldap-database-with-ldif )
• Persist /etc/ldap/slapd.d, /var/lib/ldap

here is my docker-compose.yml

version: '3'
services:
  ldap-service:
    image: osixia/openldap:1.1.8
    container_name: ldap-service
    environment:
      LDAP_ORGANISATION: example
      LDAP_DOMAIN: example.com
      LDAP_ADMIN_PASSWORD: admin
    ports:
      - "389:389"
      - "636:636"
    command: --copy-service
    volumes:
      - ldapdb:/var/lib/ldap
      - ldapconf:/etc/ldap/slapd.d
      - ./foobar.schema:/container/service/slapd/assets/config/bootstrap/schema/foobar.schema

[752319951]

@Kumassy Thank you very much, this method works!

by the way... I export whole schema file from old ldap and try to use above method but failed.. so i delete irrelevant content (maybe contents in core.schema etc.) and keep newly added content only. This process is complex for me. is there any way to use whole old schema file?

[Kumassy]

startup.sh converts .schema -> .ldif and ldapadd them, only when is_new_schema returns 1.

Writing a shell script based on these lines may work.

https://github.com/osixia/docker-openldap/blob/7ce084bab1f9016830548558241fdefb60862482/image/service/slapd/startup.sh#L138-L145

https://github.com/osixia/docker-openldap/blob/7ce084bab1f9016830548558241fdefb60862482/image/service/slapd/startup.sh#L329-L339

[kevdogg]

Sorry to labor the point, but I have existing ldap container with database -- not starting from scratch. Is it possible to add a schema in this instance? The bootstrap method seems to be for empty containers. I'd like to add ldapns.schema schema.