xiki not working under windows because fork is unimplemented.

[jarmo]

C:\Users\jarmo>xiki ip
mkfifo: fifo files not supported
mkfifo: fifo files not supported
- service couldn't start!:fork() function is unimplemented on this machine
C:/tcs-ruby193_require_fenix_gc_hash_20120527/lib/ruby/gems/1.9.1/gems/daemons-1.1.9/lib/daemons/daemonize.rb:11:in `fork'
C:/tcs-ruby193_require_fenix_gc_hash_20120527/lib/ruby/gems/1.9.1/gems/daemons-1.1.9/lib/daemons/daemonize.rb:11:in `safefork'
C:/tcs-ruby193_require_fenix_gc_hash_20120527/lib/ruby/gems/1.9.1/gems/daemons-1.1.9/lib/daemons/daemonize.rb:93:in `daemonize'
C:/tcs-ruby193_require_fenix_gc_hash_20120527/lib/ruby/gems/1.9.1/gems/daemons-1.1.9/lib/daemons/application.rb:146:in `start_load'
C:/tcs-ruby193_require_fenix_gc_hash_20120527/lib/ruby/gems/1.9.1/gems/daemons-1.1.9/lib/daemons/application.rb:298:in `start'
C:/tcs-ruby193_require_fenix_gc_hash_20120527/lib/ruby/gems/1.9.1/gems/daemons-1.1.9/lib/daemons/controller.rb:70:in `run'
C:/tcs-ruby193_require_fenix_gc_hash_20120527/lib/ruby/gems/1.9.1/gems/daemons-1.1.9/lib/daemons.rb:147:in `block in run'
C:/tcs-ruby193_require_fenix_gc_hash_20120527/lib/ruby/gems/1.9.1/gems/daemons-1.1.9/lib/daemons/cmdline.rb:109:in `call'
C:/tcs-ruby193_require_fenix_gc_hash_20120527/lib/ruby/gems/1.9.1/gems/daemons-1.1.9/lib/daemons/cmdline.rb:109:in `catch_exceptions'
C:/tcs-ruby193_require_fenix_gc_hash_20120527/lib/ruby/gems/1.9.1/gems/daemons-1.1.9/lib/daemons.rb:146:in `run'
C:/tcs-ruby193_require_fenix_gc_hash_20120527/lib/ruby/gems/1.9.1/gems/xiki-0.6.3/etc/command/xiki_command.rb:87:in `run'
C:/tcs-ruby193_require_fenix_gc_hash_20120527/lib/ruby/gems/1.9.1/gems/xiki-0.6.3/bin/xiki:30:in `<top (required)>'
C:/tcs-ruby193_require_fenix_gc_hash_20120527/bin/xiki:23:in `load'
C:/tcs-ruby193_require_fenix_gc_hash_20120527/bin/xiki:23:in `<main>'

The problem is that fork is not available on Windows platform. The problem could be solved by using fork-like techniques on Windows. One way to do that cross-platform way would be to use the gem childprocess.

[trogdoro]

The problem is that fork is not available on Windows platform. The problem could be solved by using fork-like techniques on Windows.

Yeah, I figured a windows-savvy person would suggest something. The requirement is basically:

• send text back and forth with a process
  • which I used the 'daemons" gem to start up
• allow only users with sufficient permissions
  • important because xiki can read/write arbitrary files
  • with pipes, the file permission on them blocks unprivileged users
  • DRb allows any user to connect, so wouldn't be super secure

One way to do that cross-platform way would be to use the gem childprocess.

Cool! Wasn't aware of that gem. Can you use it to talk to processes started up by the daemons gem? If not, does it provide a way to launch a process that stays alive even though the main process died (so the main process can start up again later and talk to the process again)? Does it provide a way to block unprivileged users?

If it satisfies the requirements, could you paste a simple code example (or link) of using it to talk to a process?

--Craig

[jarib]

Not sure about the daemons gem, but you can still communicate with pipes, so setting permissions on them should work fine.

See these specs:

https://github.com/jarib/childprocess/blob/master/spec/io_spec.rb#L57 https://github.com/jarib/childprocess/blob/master/spec/io_spec.rb#L88

You can also quite easily detach from the parent process:

 process = ChildProcess.build(*args)
 process.detach = true
 process.start

[trogdoro]

Are you saying this will work with Windows?

--Craig

On Mon, Sep 17, 2012 at 4:36 AM, Jari Bakken [email protected]:

Not sure about the daemons gem, but you can still communicate with pipes, so setting permissions on them should work fine.

See these specs:

https://github.com/jarib/childprocess/blob/master/spec/io_spec.rb#L57 https://github.com/jarib/childprocess/blob/master/spec/io_spec.rb#L88

You can also quite easily detach from the parent process:

process = ChildProcess.build(*args) process.detach = true process.start

— Reply to this email directly or view it on GitHubhttps://github.com/trogdoro/xiki/issues/27#issuecomment-8611497.

[jarib]

Yeah, though I'm not sure how setting permissions on pipes will work on Windows. If you write a short snippet that demonstrates your use case with pipe, fork and exec, I'm sure @jarmo will help you write and test a childprocess version on Windows :)

[jarmo]

@trogdoro as @jarib said - if you can create a patch, which you think should work on all platforms then i would be willing to try it out :)

[trogdoro]

A guy in another thread submitted a patch that may get Xiki working using cygwin! Will look at his pull request soon.

[jarmo]

I'm not using cygwin and there are many windows users who do not use it so that would not solve the problem for me :(

[trogdoro]

It's been a while since I used Cygwin, but I remember it being relatively easy to set up and pretty unobtrusive. Ever tried it (and if so how did you find the experience)?

--Craig

On Wednesday, September 19, 2012, Jarmo Pertman wrote:

I'm not using cygwin and there are many windows users who do not use it so that would not solve the problem for me :(

— Reply to this email directly or view it on GitHubhttps://github.com/trogdoro/xiki/issues/27#issuecomment-8683999.

[jevan]

I'm very interested in non-cygwin windows support too! Cygwin is easy to install... I used to have it on pretty much all my window machines... but I started running into conflict issues when using any other programs/packages that included their own cygwin.

[jarmo]

@trogdoro it is relatively easy to set up indeed, but does not work well with everything natively on Windows which means that i have to use regular terminal + cygwin terminal. That does not make it really useful and/or user friendly and i have been ending up using unxutils instead - so i get all the nice unix utilities and none of cygwin's problems. That's why i prefer a solution without using cygwin.

[tylorr]

I have the same problem as the OP is there any solution in the works for this?

[trogdoro]

What are you trying to do? If you're trying to just use Xiki with emacs, the fork error won't get in your way (it's used by the "xiki" shell command). You can just skip that part of the instructions. I don't know whether it will work from that point onward on Windows, but it may.

--Craig

On Wed, Nov 21, 2012 at 4:30 PM, Tylor Reynolds [email protected] wrote:

I have the same problem as the OP is there any solution in the works for this?

— Reply to this email directly or view it on GitHub.

[tylorr]

I'm using Sublime Xiki and the only things that I can do are explore file hierarchies and run other command line programs such as git, but I can't do much else. I don't know if this is a problem with Sublime Xiki, but I do know that I can't even run Xiki on the command line without getting an error about fork not being supported.

[caoimhinp]

Same issue as above comment without the git functionality. Yes, I have git installed and it works from the command line. Mostly I get a "The system cannot find the file specified" error.

Trogdoro, please just be straight so I and others can make a decision to remain interested or not. Do you actually care about Windows support without use of Emacs? Most signs indicated that you don't but there are occasional head nods to Windows. I'm not a Windows fan but I do my development in Windows because I develop for Windows. Nothing I can change about that. I think there is some great functionality here that I would love to use. I found xiki by way of Sublime plugins. The combination would be rather comfortable.

Thanks for your attention.

[trogdoro]

Sorry for not being responsive. I've been busy with another project.

Trogdoro, please just be straight so I and others can make a decision to remain interested or not. Do you actually care about Windows support without use of Emacs?

Yeah, there's no reason Xiki shouldn't work on all platforms eventually. To get there it will take windows devs who are willing to contribute and make it happen. Here are some lines in the Xiki source that are dependent on commands that aren't in windows by default (without cygwin):

/projects/xiki/etc/command/

• xiki_command.rb | mkfifo -m 600 /tmp/xikirequest if ! File.exists?("/tmp/xikirequest") | open("/tmp/xikirequest", 'w+') do |out|
• xiki_process.rb | open('/tmp/xikirequest', 'r+') do |f|

The 1st line creates a socket. The 2nd line is for writing to it. The 3rd line is for reading from it.

This is how the xiki command talks to the xiki process. The xiki process exists so it can make calls to the xiki command fast, by staying alive to avoid the startup code running more than once.

The lines of code dealing with "xikiresponse" are similar - I left them out for brevity.

So, Windows devs... what's a way to do this that would be really fast, cross-platform (if possible) and not open up a security hole by allowing unprivileged users to call xiki commands (since can modify the filesystem as the user running the process, which will normally be your user account)? There are many ways of communicating between processes, but many of them (drb, etc) are lax security-wise. They let any process make a connection (to my understanding), whereas with the above approach your user has to have write permissions, or you won't be able to write to /tmp/xikirequest.

--Craig

[caoimhinp]

Trogdoro, thanks for answering. I'm not a big ruby dev (nor Windows dev to be honest - which is why I stick to python on Windows) but I'll talk to some folks and see if I can get some assistance with assisting you.

I really like the project so I'm interested in being able to use it wherever I may find myself.

caoi

[wernight]

@trogdoro: On Windows the security is normally per process based on elevation and account launching the process. This means that for normal users, the Xiki process should simply be launched by that user with the same elevation. For users with elevated privileges (Admin), same thing. The only canvas is that if you use a file as communication system, that file must be read/write only for current users and Admins (if elevated). It should be in the users' temp folder also which is what Xiki seems to do already.

As for other means of communication, named pipes may be good and pretty cross platform (similar tech, different implementation). They should be secured like above by default, see http://msdn.microsoft.com/en-us/library/windows/desktop/aa365600(v=vs.85).aspx

[trogdoro]

Including [email protected] in the thread...

As for other means of communication, named pipes may be good and pretty cross platform (similar tech, different implementation). They should be secured like above by default, see http://msdn.microsoft.com/en-us/library/windows/desktop/aa365600(v=vs.85).aspx

Cool! This seems like it could work interchangeably with how the 'xiki' command talks to the daemon. Can you paste a simple hello world working example that does this?

1. How to creates the pipe file
2. Code that writes to it
3. Code that reads from it

Also, what's the latest best practice for testing out Windows stuff from a mac?

If it's cross-platform-ish, maybe I could implement it on the Mac, and expect it to just work on Windows?

--Craig

[wernight]

A sample could be found on http://stackoverflow.com/questions/9796258/how-do-i-block-on-reading-a-named-pipe-in-ruby

As for testing, a VM (VMware or similar) should be fine. If not, I'm sure people can test it, including myself.

[trogdoro]

As for other means of communication, named pipes may be good

According to that page, Xiki already is using named pipes (search the Xiki codebase for "mkfifo" and you'll see that's what it's doing).

A sample could be found on http://stackoverflow.com/questions/9796258/how-do-i-block-on-reading-a-named-pipe-in-ruby

There are several samples on that page, which one do you have in mind? The one that uses "mkfifo" looks like it requires cygwin. If you have cygwin I think the current xiki implementation might work (possibly with some tweaks _____pull request). But several people have given the no cygwin

There's a lot of code on the http://stackoverflow.com/questions/9796258/how-do-i-block-on-reading-a-named-pipe-in-ruby page between the two samples starting with...

module PipeLogger ...

...and...

require 'rubygems' require 'win32/pipe' include Win32 ...

Is all of that stuff in .setup_pipe really necessary?

Perhaps http://rubydoc.info/gems/win32-pipe/0.3.0/file/README is better to look at, and has some of the lower-level stuff abstracted out? It doesn't appear to be cross-platform. But I guess having a XikiWindowsAdapter-ish class that just does it differently wouldn't be too bad.

Does the example on the above win32-pipe readme block users from reading and writing to the pipe that don't have permission for the pipe, do you know?

--Craig

As for testing, a VM should be fine. If not, I'm sure people can test it, including myself.

— Reply to this email directly or view it on GitHub.

[wernight]

@trogdoro There are two answer, I had in mind the one with 2 (top) votes.

I don't use Cygwin usually, find msys just so much better. However first my issue is with fork() not implemented (it's not on Windows). May be the pipe is actually working.

[trogdoro]

Dang, those "________" lines were my way of reminding myself to finish those sentences before sending. Obviously that didn't work :) I meant to say...

If you have cygwin I think the current xiki implementation might work (possibly with some tweaks from https://github.com/trogdoro/xiki/pull/32). But several Windows people have given the feedback that having cygwin as a Xiki dependency would make them unhappy.

@trogdoro There are two answer, I had in mind the one with 2 (top) votes. I don't use Cygwin usually

His description includes "Note that I'm using mkfifo from cygwin".

However first my issue is with fork() not implemented (it's not on Windows).

Ah, that's happening in the Daemons gem, which Xiki uses to start up a Xiki instance so subsequent calls (connected to via pipe) will be fast.

Any of you Windows people know of a Daemons gem alternative that works on Windows?

Does the example on the above win32-pipe readme block users from reading and writing to the pipe that don't have permission for the pipe, do you know?

If anyone knows this, I'd love to hear the answer.

--Craig

[wernight]

Under Windows you may have services or just a process running in background. Which one are you trying to achieve? I guess the background process. The simplest is just to spawn a new process and not wait for it to terminate.

[marcelotto]

Also, what's the latest best practice for testing out Windows stuff from a mac?

@trogdoro Although I used it lately only for the other way around (testing Unix stuff from Windows), Vagrant may be a good and free solution. It's a Ruby wrapper around VirtualBox. The list of available Vagrant boxes also lists Windows Server 2008 R2, so you could have a free Windows for tests.

[trogdoro]

process. The simplest is just to spawn a new process and not wait for it to terminate.

It would be very cool if someone would code that up a simple example in ruby and just post it to the list. It could be just starting the process and writing to it and reading from it.

If you do that, remember the implementation has to not open up security holes by letting any unprivileged process connect to it (because that will mean running arbitrary commands and code as the user who started the process).

--Craig

[wernight]

Spawn a process and communicate via some pipe read and write, both directions? On Mar 9, 2013 9:52 PM, "trogdoro" [email protected] wrote:

process. The simplest is just to spawn a new process and not wait for it to terminate.

It would be very cool if someone would code that up a simple example in ruby and just post it to the list. It could be just starting the process and writing to it and reading from it.

If you do that, remember the implementation has to not open up security holes by letting any unprivileged process connect to it (because that will mean running arbitrary commands and code as the user who started the process).

--Craig

— Reply to this email directly or view it on GitHubhttps://github.com/trogdoro/xiki/issues/27#issuecomment-14671487 .

[wernight]

I found and tested http://devver.wordpress.com/2009/10/12/ruby-subprocesses-part_3/ (see also part 1 and 2 links on that page). 5a and 5b are not working on Windows (that is forking to a pipe). However Method 4: Opening a pipe seems to work fine (described in part 2 and in full source code).

Once you have some code, I may check the permissions are respected. I guess you'd want user A (not elevated/admin) not able to read/write to pipe of user B, and not able to read/write to pipe of user A (elevated).

[trogdoro]

Thanks for researching this!

I think that's likely not a fit though. The xiki process is a separate background process that needs to stay open across calls to the "xiki" shell command.

The code in your link deals with subprocesses. It starts a new process and holds a reference to it to talk to it. Presumably the subprocess dies when the parent process dies.

The xiki shell command flow is:

• User runs "xiki ip" (for example) on command line
  • If no background process running, it starts one
    • tells the background process to invoke "ip" menu
    • the command process dies, the background process stays alive
  • Else, it uses the existing process to invoke "ip" menu
    • then dies

--Craig

On Mon, Mar 11, 2013 at 4:07 AM, Werner Beroux [email protected] wrote:

I found and tested http://devver.wordpress.com/2009/10/12/ruby-subprocesses-part_3/ (see also part 1 and 2 links on that page). 5a and 5b are not working on Windows (that is forking to a pipe). However Method 4: Opening a pipe seems to work fine (described in part 2 and in full source code).

— Reply to this email directly or view it on GitHub.

[wernight]

Code:

# gem install win32-pipe
# gem install win32-mutex
require 'win32/mutex'
require 'rbconfig'
require 'win32/pipe'
include Win32

$stdout.sync = true

THIS_FILE = File.expand_path(__FILE__)
RUBY = File.join(Config::CONFIG['bindir'], Config::CONFIG['ruby_install_name'])

def child_main(source)
  puts "[child] Started \"#{source}\""
  # Single instance (could also use a temporary file but this is a better way on Windows)
  Win32::Mutex.new(false, 'some_unique_id') do |mutex|

    while true do
      pipe_server = Pipe::Server.new("foo_pipe")
      pipe_server.connect
      data = pipe_server.read
      puts "[child] Got #{data} from client"
      pipe_server.close
    end

    mutex.release
  end
  puts "[child] Terminated"
end

if $PROGRAM_NAME == __FILE__
  cmd = %Q<#{RUBY} -r#{THIS_FILE} -e 'child_main("ip")'>
  begin
      Win32::Mutex.open('some_unique_id') do |mutex|
        mutex.close
      end
      puts "[parent] Process already running"
  rescue #Win32::Mutex::Error => e
    puts "[parent] Starting process"
    Process.spawn(cmd);
  end

  # TODO: Wait for mutex or find a better way to wait for the process to be ready.
  sleep(0.5)

  pipe_client = Pipe::Client.new("foo_pipe")
  pipe_client.write("Hello World")
  pipe_client.close
end

Execution:

$ ruby test.rb
[parent] Starting process
[child] Started "ip"
[child] Got ["Hello World"] from client

$ ruby test.rb
[parent] Process already running
[child] Got ["Hello World"] from client

[trogdoro]

Cool!

Will this deny clients who try to connect and don't have sufficient privileges?

TODO: Wait for mutex or find a better way to wait for the process to be ready.

sleep(0.5)

I wonder if a better way to wait was found. Half a second delay is way to slow. The xiki command using unix pipes completes in under .03 seconds.

Have you experimented with having smalle sleep times? I'm guessing it might start hogging the cpu if you make the sleep time down in that range?

Anyone know of a Windows equivalent to the Daemons gem? It lets you start, stop, restart and get the status of a process.

--Craig

On Wed, Mar 13, 2013 at 4:55 AM, Werner Beroux [email protected] wrote:

Code:

gem install win32-pipe

gem install win32-mutex

require 'win32/mutex' require 'rbconfig' require 'win32/pipe' include Win32

$stdout.sync = true

THIS_FILE = File.expand_path(FILE) RUBY = File.join(Config::CONFIG['bindir'], Config::CONFIG['ruby_install_name'])

def child_main(source) puts "[child] Started "#{source}""

Single instance (could also use a temporary file but this is a better way on Windows)

Win32::Mutex.new(false, 'some_unique_id') do |mutex|

while true do
  pipe_server = Pipe::Server.new("foo_pipe")
  pipe_server.connect
  data = pipe_server.read
  puts "[child] Got #{data} from client"
  pipe_server.close
end

mutex.release

end puts "[child] Terminated" end

if $PROGRAM_NAME == FILE cmd = %Q<#{RUBY} -r#{THIS_FILE} -e 'child_main("ip")'> begin Win32::Mutex.open('some_unique_id') do |mutex| mutex.close end puts "[parent] Process already running" rescue #Win32::Mutex::Error => e puts "[parent] Starting process" Process.spawn(cmd); end

TODO: Wait for mutex or find a better way to wait for the process to be ready.

sleep(0.5)

pipe_client = Pipe::Client.new("foo_pipe") pipe_client.write("Hello World") pipe_client.close end

Execution:

$ ruby test.rb [parent] Starting process [child] Started "ip" [child] Got ["Hello World"] from client

$ ruby test.rb [parent] Process already running [child] Got ["Hello World"] from client

— Reply to this email directly or view it on GitHub.