Davix Documentation

Author

Devresse Adrien ( adrie.nosp@m.n.de.nosp@m.vress.nosp@m.e@ce.nosp@m.rn.ch )

Developped at CERN (IT-SDC-ID)

Official WebSite: Here

User Documentation: Here

Mailing list : davix.nosp@m.-dev.nosp@m.el@ce.nosp@m.rn.c.nosp@m.h

davix API :

file / object store API : Davix::DavFile
posix compatiblity layer : Davix::DavPosix
http request layer: Davix::HttpRequest
main header: davix.hpp

What is davix ?

davix is a library and a set of tools for remote I/O on resources with HTTP based protocols. It aims to be a simple, performant and portable I/O layer for Cloud and Grid Storages services.

Davix supports:

• HTTP, WebDav, Amazon S3
• SSL/TLS
• X509 client auth with proxy credential support
• Vector operations (Partial reads, multi-range, single range)
• Metalinks
• Redirections caching
• Webdav parsing
• Data management operations ( mkdir, rm, stat )

The Davix targets to:

• Be simple to use for simple use cases
• Provide the needed features for High Performance I/O use cases
• Be a data management swiss knife for HTTP based data stores

Examples

Query basic file metadata

StatInfo infos;
DavFile file(context, "http://my.webdav.server.org/myfolder/myfile");
file.statInfo(NULL, infos);
std::cout << "my file is " << infos.size << " bytes large " << std::endl;

Create a directory

DavFile file(context, "http://my.webdav.server.org/myfoldier/newfolder");
// creat directory
file.makeCollection(NULL);

Get a full file content

DavFile f(context, "http://mysite.org/file");
int fd = open("/tmp/local_file", O_WRONLY | O_CREAT);
// get full file
file.getToFd(NULL,fd, NULL) < 0)

Execute a partial GET

char buffer[255] = {0};
DavFile file(context, "http://mysite.org/file");
// get 100 bytes from http://mysite.org/file after an offset of 200 bytes
file.readPartial(NULL, buffer, 100, 200);

Execute a Vector Operation

char buffer[255] = {0}
DavFile file(context, "http://mysite.org/file");


DavIOVecInput in[3];
DavIOVecOutput ou[3];
// setup vector operations parameters
// --------
// execute query
file.readPartialBufferVec(NULL, in, out , 3 , NULL);

Random I/O in posix mode

Davix::DavPosix p;

// read ops
fd= p.open(NULL, "https://mywebdav-server.org/myfile.jpg", O_RDONLY, NULL);
p.read(fd, buffer, size, NULL);
p.pread(fd, buffer, size2, offset, NULL);
p.close(fd);
//

Manual HTTP query:

Davix::HttpRequest req("https://restapi-server.org/rest")
req.addHeaderField(...)
req.setRequestMethod("PUT")
// .. configure ....
//
//
// execute your request
req.executeRequest(...);

How to compile

• Davix Dependencies :
  • openssl
  • libxml-2.0
  • Doxygen ( optional, for documentation generation )
• Davix Portability :
  • Target any POSIX OS
  • Ported on Linux > 2.6 , Windows with Cygwin, OSX > 10.2, AIX.
  • Packaged on Debian > 6, Ubuntu > 13.04, Fedora > 18, SL > 5,
• Compile :
  • " 1. git clone http://git.cern.ch/pub/davix "
  • " 2. cd davix "
  • " 3. mkdir build; cd build"
  • " 4. cmake ../"
  • " 5. make "
• Generate doc :
  • make doc
• Compile and run unit tests :
  • cmake -DUNIT_TESTS=TRUE ../
  • make
  • make test
• Compile & execute func tests : warning : functionals test needs davserver and a valid credential
  • " 4. cmake -DFUNCTIONAL_TESTS=TRUE ../ "
  • " 5. . ../test/setup_test_env.sh - " 5. make; make test"
• make RPMS :
  • ./packaging/bin/packager_rpm.sh ./packaging/rpm/specs/ ./
  • mock -r [mycfg] RPMS/davix-[...].src.rpm

Play with davix command line tool :

davix has a set of command line tools for testing purpose and demonstration

-> davix-ls: file listing
-> davix-get: download operations
-> davix-put: upload operations
-> davix-http: low level query composition

TODO in Davix

- WebHDFS support
- S3 ACL support
- CDMI support
- X-stream mode support ( metalink multi source download )

For any contribution please contact us on davix-devel@cern.ch