groovy-vfs

groovy-vfs copied to clipboard

= Groovy VFS

A DSL for Groovy to wrap around the link:http://commons.apache.org/proper/commons-vfs/[Apache Commons VFS] libraries.

If you like it, then tweet about it using #groovyvfs as the hashtag.

image:http://img.shields.io/travis/ysb33r/groovy-vfs/master.svg["Build Status", link="https://travis-ci.org/ysb33r/groovy-vfs"] image:https://ci.appveyor.com/api/projects/status/github/ysb33r/groovy-vfs?svg=true["Windows Build Status", link="https://ci.appveyor.com/project/ysb33r/groovy-vfs"]

== Groovy Library

image:https://api.bintray.com/packages/ysb33r/grysb33r/groovy-vfs/images/download.png["Download', link="https://bintray.com/ysb33r/grysb33r/groovy-vfs/_latestVersion"]

[source,groovy]

@Grapes([ @Grab( 'org.ysb33r.groovy:groovy-vfs:1.0.1' ), @Grab( 'commons-net:commons-net:3.+' ), // If you want to use ftp @Grab( 'commons-httpclient:commons-httpclient:3.1'), // If you want http/https @Grab( 'com.jcraft:jsch:0.1.48' ) // If you want sftp ]) import org.ysb33r.groovy.dsl.vfs.VFS

def vfs = new VFS()

// Simple copy operation vfs.cp 'ftp://foo.example/myfile', 'sftp://bar.example/yourfile'

// Utilising the DSL vfs {

// Copy file from one site to anther using two different protocols
cp 'http://first.example/myfile', 'sftp://second.example/yourfile'

// Not implemented yet - move file between two sites using different protocols
mv 'sftp://second.example/yourfile', 'ftp://third.example/theirfile'

// Lists all files on a remote site
ls ('http://first.example') {
    println it.name
}

// Streams the output
cat ('http://first.example/myfile') { strm->
    println strm.text
}

// Create a new folder on a remote site
mkdir 'sftp://second.example/my/new/folder'

// Change default options via property Map
options 'vfs.ftp.passiveMode' : true

// Change default options DSL style
options {
    ftp {
        passiveMode true
    }
}

// Use options on a per URL basis
cp 'ftp://first.example/myfile?vfs.ftp.passiveMode=1', 'sftp://second.example/yourfile?vfs.sftp.compression=zlib'

// Download a compressed archive and unpack to local directory
cp 'tbz2:ftp:/first.example/myFiles.tar.bz2", new File( '../unpack-here' ), recursive:true

// Replace content of a file with text
overwrite 'ftp://first.example/myfile?vfs.ftp.passiveMode=1' with 'this text'
overwrite 'ftp://first.example/myfile?vfs.ftp.passiveMode=1', { strm -> strm << 'this text' }

// Append content to a file
append 'ftp://first.example/myfile?vfs.ftp.passiveMode=1' with 'this text'
append 'ftp://first.example/myfile?vfs.ftp.passiveMode=1', { strm -> strm << 'this text' }

}

== Gradle plugin

image:https://api.bintray.com/packages/ysb33r/grysb33r/vfs-gradle-plugin/images/download.png["Download", link="https://bintray.com/ysb33r/grysb33r/vfs-gradle-plugin/_latestVersion"]

From initiation a VFS object has been available as an extension to the project class. The interface is very experimental and may change without much warning in future releases of this plugin.

[source,groovy]

buildscript { repositories { jcenter() mavenCentral() } dependencies { classpath 'org.ysb33r.gradle:vfs-gradle-plugin:1.0.1' classpath 'commons-net:commons-net:3.+' // If you want to use ftp classpath 'commons-httpclient:commons-httpclient:3.1' // If you want http/https classpath 'com.jcraft:jsch:0.1.48' // If you want sftp } } apply plugin : 'org.ysb33r.vfs'

// Create a VFS task task copyReadme << { vfs { cp 'https://raw.github.com/ysb33r/groovy-vfs/master/README.md', new File("${buildDir}/tmp/README.md") } }

// it is also possible to update global options for vfs vfs { options { http { maxTotalConnections 4 } } }

If you want to see what VFS is going run gradle with --debug

In 1.0 a copy task has been added.

[source,groovy]

import org.ysb33r.gradle.vfs.tasks.VfsCopy

task download ( type : VfsCopy ) {

from 'http://somewhere.example/file' <1>

from URL_2

// Set the destination root URI. Files will be copied into that folder
into URL_ROOT

// Set options for all of the copy operations.
// These override global options, but are only applicable for source and destination URLs within this task
options {
  ftp {
    passiveMode true
  }
}

}

<1> Copy the file at this URI <2>

Any local source URIs will get reflected as an input file in the TaskInputs, otherwise it is just an input If the destination URI is local, it will get reflected as TaskOutputs as a file

== Adding extra plugins

From v1.0 onwards additional plugins can be loaded via a new extend block. For more details see this gist: https://gist.github.com/ysb33r/9916940

== SMB provider

image:https://api.bintray.com/packages/ysb33r/grysb33r/groovy-vfs-smb-provider/images/download.png["Download', link="https://bintray.com/ysb33r/grysb33r/groovy-vfs-smb-provider/_latestVersion"]

A provider for accessing SMB shares is now avavilable. The plugin must be loaded separately.

[source,groovy]

@Grab( 'org.ysb33r.groovy:groovy-vfs-smb-provider:1.0-beta1' ), @Grab( 'jcifs:jcifs:1.3.17' ),

vfs { extend { provider className: 'org.ysb33r.groovy.vfsplugin.smb.SmbFileProvider', schemes: ['smb','cifs'] }

cp 'smb://someserver/share/dir/file', new File('localfile.txt) }

NOTE: when embedding windows credentials in the URL use %5C in place of backslash i.e.

smb://DOMAIN%5cUSERNAME:PASSWORD@HOSTNAME/SHARE/PATH

NOTE: This provider has a concurrency bug (https://github.com/ysb33r/groovy-vfs/issues/73[#73]).

== S3 provider (EXPERIMENTAL) image:https://api.bintray.com/packages/ysb33r/grysb33r/groovy-vfs-cloud-core/images/download.png["Download', link="https://bintray.com/ysb33r/grysb33r/groovy-vfs-cloud-core/_latestVersion"]

A provider for accessing S3 shares is now available and will be fully supported in future version. The plugin must be loaded separately.

[source,groovy]

@Grab( 'org.ysb33r.groovy:groovy-vfs-cloud-core:0.1-beta1' ), @Grab( 'org.apache.jclouds:jclouds-all:1.7.2' ) @Grab( 'org.apache.jclouds.driver:jclouds-jsch:1.7.2' ) @Grab( 'org.apache.jclouds.provider:aws-s3:1.7.2' vfs { extend { provider className: 'org.ysb33r.groovy.vfsplugin.cloud.s3.S3FileProvider', schemes: ['s3'] }

cp 'smb://id:key@bucket/dir/file', new File('localfile.txt) }

NOTE: Although S3 does not actually support folders, this is simulated through the use of folder names containing / characters.

== Command-line utility image:https://api.bintray.com/packages/ysb33r/nanook/vfs/images/download.png["Download', link="https://bintray.com/ysb33r/nanook/vfs/_latestVersion"]

A command-line utility mimicking a number of GNU shell utilities is available.

== Documentation

• See https://github.com/ysb33r/groovy-vfs/wiki for more detailed documentation.
• Greach2014 presentation on v0.5 - http://www.slideshare.net/ysb33r/groovy-vfs-32889561
• GGX2014 presentation on v0.5 & v1.0 - https://skillsmatter.com/skillscasts/6049-groovy-vfs

== Credits

It is seldom that these kind of libraries happen in isolation. It is therefore prudent that I acknowledge the inputs of others in the creation of groovy-vfs

• Luke Daley (https://gist.github.com/alkemist/7943781) for helping to use Ratpack as a Mock HTTP Server in unit tests.
• Will_lp (https://gist.github.com/will-lp/5785180) & Jim White (https://gist.github.com/jimwhite/5784982) offered great suggestions when I got stuck with the config DSL.
• Jez Higgins, Rob Fletcher, Giovanni Asproni, Balachandran Sivakumar, Burkhard Kloss & Tim Barker who helped shape the design decision to auto-create intermediates during a move operation.
• Maarten Boekhold for testing the SMB Provider plugin
• Everyone from Greach 2014 that provided feedback