mssql-docker icon indicating copy to clipboard operation
mssql-docker copied to clipboard

SSISDB/SSIS in docker linux containers?

Open scoobs76 opened this issue 7 years ago • 57 comments

I was wondering if there are any solutions or sample docker files available that will get SSIS loaded to a SQL Server image for running on a linux cointainer? I began exploring "https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-setup-ssis", but these scripts don't appear to be able to run via a Dockerfile. Please advise if I am wrong. Thanks.

scoobs76 avatar Dec 05 '17 20:12 scoobs76

Integration services is not yet supported in a container.

twright-msft avatar Dec 06 '17 02:12 twright-msft

Any time estimates for this support? Thanks.

scoobs76 avatar Dec 06 '17 16:12 scoobs76

Not in the near term.

twright-msft avatar Dec 06 '17 22:12 twright-msft

We just came across this GitHub post...

We were able to run the same scripts for installing SSIS in a Linux Docker container. Although at the end it does state the error: "Failed to get D-Bus connection: Operation not permitted Attempting to start the telemetry service failed."

Running dtexec was working ok in our tests.

Though, now after reading the above, we are considering that we will need to deploy in VMs instead.

We have searched through several Microsoft documentation, including the SQL Server 2017 white paper, looking for an indication and details that SQL Server Integration Services is not supported in a Docker container, but have not found any.

Can you please give some more details on SSIS not being supported in Docker containers, or point us to documentation stating so. We would rather have more to reference than this GitHub post, for the change in deployment to VMs.

Many thanks!

YodaNinja avatar Mar 29 '18 16:03 YodaNinja

The absence of something stating it is supported in documentation means it's not supported. (yet) :) If there is enough demand for this we can talk to the SSIS team about trying to get it done though. If anybody else is interested in this, please thumbs up here and we'll loop in the SSIS team.

twright-msft avatar Mar 29 '18 16:03 twright-msft

If SSIS was supported on containers, would that be both windows and linux containers? We have use cases for both. Thanks.

mikesurel avatar Apr 25 '18 20:04 mikesurel

@mikesurel - TBD. I see here that 8 people have upvoted this now. I'll shoot an email over to the SSIS team letting them know about the interest here.

twright-msft avatar Apr 26 '18 01:04 twright-msft

I'd love to convince my office to move SSIS into a container so I can just have them spin up and down as packages needed to fire on a schedule. Or something along those lines.

ghost avatar Apr 27 '18 17:04 ghost

Containerized SSIS would be a significant benefit for our organization.

warrenrumak avatar Apr 30 '18 19:04 warrenrumak

We are waiting for SSIS containerization because it will be next big move for most of the organizations. Hope SSIS Team started working on it. If not, requesting to engage them.

Thanks In Advance.

TayiSaiKishore avatar May 07 '18 07:05 TayiSaiKishore

We are waiting for SSIS containerization too.

dentaku65 avatar Jul 13 '18 05:07 dentaku65

This would be a huge step. We're relying on server pools running SSIS, containerization will take it that one step further to abstract the workloads.

nsbradley88 avatar Jul 21 '18 06:07 nsbradley88

In terms of scalability, it would be a huge step forward in using SSIS. At the moment we have SSIS in an out-scaling configuration on VMs. Unfortunately, this does not give the ability to automatically scale out when more resources are required. Containerizing SSIS will make an out-scaling configuration so much more powerful.

AHulshoff avatar Jul 26 '18 15:07 AHulshoff

Hi,

I don't want to bug you with this topic but Do you know if SSIS will be available in Containers this year?

Thanks!!

luismiguelsanroman avatar Aug 24 '18 15:08 luismiguelsanroman

We generally operate in 6 month planning/dev cycles. The work to put SSIS in containers is not committed by the SSIS team for this 6 month period which ends at the end of the year. Planning for the next 6 months will start in another couple of months so now is a good time to start building up the demand and use cases for it! :) I dont see a feature request for SSIS in containers over on the SQL Server User Voice site yet so it might be a good idea to get this request logged over there and get some additional upvotes and input there. https://feedback.azure.com/forums/908035-sql-server

twright-msft avatar Aug 24 '18 15:08 twright-msft

I work for a large bank, and we need this in the baddest way. We are a big MS shop and have thousands of SSIS packages we ought to put into GitLab pipelines. Including SSIS in a Docker images would make this far easier.

MFMorrissey avatar Aug 24 '18 21:08 MFMorrissey

Hi @twright-msft , I'm sharing the feature request from UserVoice that @MFMorrissey created. https://feedback.azure.com/forums/908035-sql-server/suggestions/35208061-add-ssis-to-the-sql-server-docker-image Anyone interested, please upvote. This will be an immensely game changing feature 👍

kethahel99 avatar Aug 28 '18 08:08 kethahel99

I got SSIS working inside Docker by modifying the two Python scripts that attempt to start/stop telemetry via systemctl which doesn't work in a container. I wish I could submit a PR for make those two files container aware as the changes are trivial in the end. You can simply remove all telemetry code and it works, however I do NOT condone this as only the paid versions can be excepted from telemetry per the current scripts.

thezim avatar Sep 17 '18 20:09 thezim

I got SSIS working inside Docker by modifying the two Python scripts that attempt to start/stop telemetry via systemctl which doesn't work in a container. I wish I could submit a PR for make those two files container aware as the changes are trivial in the end. You can simply remove all telemetry code and it works, however I do condone this as only the paid versions can be excepted from telemetry per the current scripts.

Can you share the modifications?

vrxj81 avatar Oct 08 '18 12:10 vrxj81

I got SSIS working inside Docker by modifying the two Python scripts that attempt to start/stop telemetry via systemctl which doesn't work in a container. I wish I could submit a PR for make those two files container aware as the changes are trivial in the end. You can simply remove all telemetry code and it works, however I do condone this as only the paid versions can be excepted from telemetry per the current scripts.

Hi thezim, can you please share the modification that you did to get SSIS working inside container ????

mcsreddi avatar Jan 15 '19 18:01 mcsreddi

I got SSIS working inside Docker by modifying the two Python scripts that attempt to start/stop telemetry via systemctl which doesn't work in a container. I wish I could submit a PR for make those two files container aware as the changes are trivial in the end. You can simply remove all telemetry code and it works, however I do condone this as only the paid versions can be excepted from telemetry per the current scripts.

I'm also very curious to know the details of your fix, please report back if you can!

BradMJustice avatar Mar 19 '19 16:03 BradMJustice

Greetings Brad,

That's awesome. However, I am thinking for Vulernability view for securing coding of these python scripts from OWASP standards.

Correct me, if this doesn't make any sense here?

Thanks, Sai

Get Outlook for Androidhttps://aka.ms/ghei36


From: BradMJustice [email protected] Sent: Tuesday, March 19, 2019 9:59:30 PM To: Microsoft/mssql-docker Cc: Sai Tayi; Comment Subject: Re: [Microsoft/mssql-docker] SSISDB/SSIS in docker linux containers? (#213)

I got SSIS working inside Docker by modifying the two Python scripts that attempt to start/stop telemetry via systemctl which doesn't work in a container. I wish I could submit a PR for make those two files container aware as the changes are trivial in the end. You can simply remove all telemetry code and it works, however I do condone this as only the paid versions can be excepted from telemetry per the current scripts.

I'm also very curious to know the details of your fix, please report back if you can!

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Microsoft_mssql-2Ddocker_issues_213-23issuecomment-2D474457326&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=lAUUuJHh46cYRvNzaenWpJN9LADXSWn04QcBwiwemPk&m=G6qrvIBVNfrWCLxfjnBGfrdOwhYgg_6-ZOhZQ_xn7_c&s=JO0QIx0aHXGs0kHxQRlULnKTJ1EjKnOcCB3MFAbxi9k&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AM6kW1QNejGqNhKhzBGxlJ-2DYnjsFFq-2Diks5vYRBpgaJpZM4Q263q&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=lAUUuJHh46cYRvNzaenWpJN9LADXSWn04QcBwiwemPk&m=G6qrvIBVNfrWCLxfjnBGfrdOwhYgg_6-ZOhZQ_xn7_c&s=sZSIDtgowxJ2Ga3NJ1L2Z_fyhGd8rTz3mo5zYcJzT68&e=.

-------------------------------------------------- Confidentiality And Disclaimer Notice: Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.netenrich.com -------------------------------------------------

TayiSaiKishore avatar Mar 19 '19 19:03 TayiSaiKishore

We are having on premise K8S cluster and using MSSQL with SSIS. Right now we have to have SQL installed separately outside of cluster because of un-ability to have SSIS running inside linux containers. So having this working would be a blessing indeed!

petermicuch avatar May 24 '19 12:05 petermicuch

This feature would be extremely helpful.

However, I am a little frustrated because this issue was closed with a reference to #2927 and that issue was closed referencing this one.

Should we make another one that references both?

maxrank avatar Oct 17 '19 18:10 maxrank

@maxrank This issue is not closed, and still active.

cilerler avatar Oct 17 '19 19:10 cilerler

We can vote for this idea using the next link https://feedback.azure.com/forums/908035-sql-server/suggestions/34912474-ssis-ssas-ssrs-linux-sql-server-runs-on-linux-w

oscarvesga avatar Dec 06 '19 17:12 oscarvesga

Has there been any updates to running SSIS on a linux container?

coltonshipley avatar Feb 13 '20 14:02 coltonshipley

Although votes for this request are rising slowly (https://feedback.azure.com/forums/908035-sql-server/suggestions/35208061-add-ssis-to-the-sql-server-docker-image) and you can still up-vote there, it seems this is still just under review for over one year. This is really a pain. We have to keep VMs in our setups just because of this and we strongly consider going away from SSIS if this will go unnoticed for longer time. We are otherwise completely migrated to K8S with our product. I get that this is not such a sexy feature of mssql as others in the wish list, which might be also caused by tiny size of SW community that is involved in here, not making it less significant speaking of investments made. So please do something about this :-)

petermicuch avatar Feb 13 '20 14:02 petermicuch

You can run SSIS in containers without hacks, just init it with systemd. The real problem is that it doesn't work with Windows Auth. They claim it should work via kerberos, but it doesn't (kerberos work for .net code and sqlcmd on the same container) The other problem is licensing - how does it apply to containers?

mikeTWC1984 avatar Jun 09 '20 20:06 mikeTWC1984

below is a sample docker file. The only thing you'd need to run that set up command after creating container

FROM mcr.microsoft.com/powershell:7.0.0-centos-7

RUN yum update -y
RUN yum install systemd python3 unzip git krb5-workstation krb5-libs vim -y 
RUN yum clean all

RUN curl -o /etc/yum.repos.d/mssql-server.repo https://packages.microsoft.com/config/rhel/7/mssql-server-2019.repo
RUN yum install -y mssql-server-is

COPY odbcinst.ini odbc.ini krb5.conf /etc/
RUN curl https://packages.microsoft.com/config/rhel/7/prod.repo > /etc/yum.repos.d/mssql-release.repo
RUN ACCEPT_EULA=Y yum install msodbcsql17 mssql-tools -y
RUN yum install unixODBC-devel -y
ENV PATH=/opt/ssis/bin:/opt/mssql-tools/bin:$PATH

ENV ODBCINI=/etc/odbc.ini

WORKDIR /root


CMD ["/usr/sbin/init"]

to run it:

 docker run --name ssis -d --privileged=true -v C:\ssis:/ssis  ssis  
 docker exec ssis bash -c 'SSIS_PID=Developer ACCEPT_EULA=Y /opt/ssis/bin/ssis-conf -n setup'

mikeTWC1984 avatar Jun 09 '20 20:06 mikeTWC1984