Project build post-provision test failure

[incredincomp]

I have been trying to get this to work for a couple weeks now on various OS's, really trying not to open issues but I am up against a wall at this point. I am now running on 18.04, and I was able to build a project finally and actually have it spit out flags and hints for me to use for our ctf. Now when trying to build the project with ruby secgen.rb build-vms --project projects/ctf/, it fails at various points. The most violent and disappointing error is below.

Using scenario flawed_fortress_1, virtual box 5.2.34, vagrant 2.2.6, on ubuntu 18.04. Anything else that I can provide that will help me help you, please let me know!

Is there a way to skip the post provisioning and test manually after the vm's are created? It hurts to see them being built, start, and then just vanish on me. My heart hurts :'(

edit: i am trying to just do vagrant up as recommended in https://github.com/cliffe/SecGen/issues/114#issuecomment-382564613, I will see how it works out... brb

Running post-provision tests...
Running tests for mysql_stretch_compatible
PASSED: mysqld process running? local command (ps -ef | grep mysqld) matches with output (/usr/sbin/mysqld) on that_escalated_quickly!
Running tests for chkrootkit
PASSED: Chkrootkit binary exists? local command (sudo ls -la /usr/sbin/chkrootkit) matches with output (chkrootkit-0.49) on that_escalated_quickly!
PASSED: Chkrootkit runs? local command (sudo /usr/sbin/chkrootkit -V) matches with output (chkrootkit version 0.49) on that_escalated_quickly!

Post provision tests contained failures!
/usr/lib/ruby/2.5.0/net/http.rb:939:in `rescue in block in connect': Failed to open TCP connection to 172.16.0.7:80 (No route to host - connect(2) for "172.16.0.7" port 80) (Errno::EHOSTUNREACH)
	from /usr/lib/ruby/2.5.0/net/http.rb:936:in `block in connect'
	from /usr/lib/ruby/2.5.0/timeout.rb:93:in `block in timeout'
	from /usr/lib/ruby/2.5.0/timeout.rb:103:in `timeout'
	from /usr/lib/ruby/2.5.0/net/http.rb:935:in `connect'
	from /usr/lib/ruby/2.5.0/net/http.rb:920:in `do_start'
	from /usr/lib/ruby/2.5.0/net/http.rb:909:in `start'
	from /usr/lib/ruby/2.5.0/net/http.rb:480:in `get_response'
	from /usr/lib/ruby/2.5.0/net/http.rb:462:in `get'
	from /home/ii/bin/SecGen/projects/ctf/lib/post_provision_test.rb:62:in `test_html_returned_content'
	from /home/ii/bin/SecGen/projects/ctf//puppet/that_escalated_quickly/modules/gitlist_040/secgen_test/gitlist_040.rb:13:in `test_module'
	from /home/ii/bin/SecGen/projects/ctf/lib/post_provision_test.rb:34:in `run'
	from /home/ii/bin/SecGen/projects/ctf//puppet/that_escalated_quickly/modules/gitlist_040/secgen_test/gitlist_040.rb:17:in `<main>'

Running tests for ntp
FAILED: unable to connect to ntp on UDP port 123
Post provision tests contained failures!


Post provision tests contained failures!
/home/ii/bin/SecGen/projects/ctf//puppet/in_the_wild/modules/parameterised_accounts/secgen_test/parameterised_accounts.rb:16:in `test_accounts_exist': undefined method `each' for nil:NilClass (NoMethodError)
	from /home/ii/bin/SecGen/projects/ctf//puppet/in_the_wild/modules/parameterised_accounts/secgen_test/parameterised_accounts.rb:12:in `test_module'
	from /home/ii/bin/SecGen/projects/ctf/lib/post_provision_test.rb:34:in `run'
	from /home/ii/bin/SecGen/projects/ctf//puppet/in_the_wild/modules/parameterised_accounts/secgen_test/parameterised_accounts.rb:24:in `<main>'

Running tests for unrealirc_3281_backdoor
FAILED: Port 6667 is closed at 172.16.0.5 (in_the_wild)!
Post provision tests contained failures!

Running tests for vsftpd
FAILED: Port 21 is closed at 172.16.0.5 (in_the_wild)!
Post provision tests contained failures!

Running tests for unrealirc
FAILED: Port 6667 is closed at 172.16.0.5 (in_the_wild)!
Post provision tests contained failures!

Running tests for distcc_exec
FAILED: Port 3632 is closed at 172.16.0.5 (in_the_wild)!
Post provision tests contained failures!

Error provisioning VMs, destroying VMs and exiting SecGen.
Loading vagrant (destroy -f) in /home/ii/bin/SecGen/projects/ctf/
==> that_escalated_quickly: Forcing shutdown of VM...
==> that_escalated_quickly: Destroying VM and associated drives...
==> in_the_wild: Forcing shutdown of VM...
==> in_the_wild: Destroying VM and associated drives...
==> decode_me: Forcing shutdown of VM...
==> decode_me: Destroying VM and associated drives...

[incredincomp]

So after following a few steps in https://github.com/cliffe/SecGen/issues/114#issuecomment-382564613, I was able to vagrant up then vagrant halt the vms, which worked as needed. Should I just be manually testing these boxes now to be sure they were provisioned properly or?

[thomashaw]

Hi @incredincomp

Yes I'd recommend manually testing them for now. You can run SecGen without the post-provision tests with the flag --no-tests. There are a few issues with the tests as it stands, they need some rework.

I will take a proper look at updating the integration testing at some point in the future. For now, I'm going to disable the tests by default as I think they're causing more issues than they're preventing. I will push a commit up in the next day or two which inverts the logic by passing a flag to enable rather than disable them.

Apologies for the trouble you've had, but thanks for letting us know.

Cheers, Tom