vboxvfs seems to have problems with mmapped access to files => retry with 3.1.4

[Jan]

I've created a VM with a clone of my webserver's configuration to test my projects under "real" conditions. The Apache webroot is a shared folder mountpoint which is connected to the folder in the host system where I store my sources for the web development.

In the guest VM, I have no problems to access the documents with cat, but the Apache doesn't deliver the files if they're larger than 255 bytes. If I reconfigure Apache to use a handler (e.g. mod_php) to process the requested file before delivering, everything works as desired.

Up to my knowledge, Apache uses mmap() for fast access to static content. Probably this mechanism needs a special support of the underlying file system which vboxvfs doesn't provide.

Host:

• Gentoo Linux (Sabayon) amd64
• Kernel 2.6.23-sabayon
• CPU: Intel(R) Core(TM)2 CPU T7200 @ 2.00GHz

Guest:

• uses VT
• Gentoo Linux x86
• Kernel 2.6.22-gentoo-r8
• Apache 2.2

[Frank Mehnert]

Actually this problem resulted from a missing implementation of the Linux sendfile system call in our VFS module. We fixed this in SVN and will backport this fix to 1.5.4 if no regressions are observed. To fix your guest, reinstall the guest additions containing this fix (SVN for now).

[Frank Mehnert]

Fixed in 1.5.4. Make sure to reinstall the guest additions.

[Martin Thielecke]

This bug still exists in 2.1.0 (Host: Ubuntu Intrepid) with a fresh installed Debian 4.0 as guest.

The kernel on the guest is 2.6.18-6-686 (Debian 4.0 standard) and Apache is 2.2.3-4+etch6. I'm using a workaround for apache now, but I think, this bug should be fixed :)

Set following option as workaround for Apache:

EnableMMAP Off

[Martin Thielecke]

Forgot the second option for Apache, so please use following workaround: EnableSendfile Off EnableMMAP Off

[Frank Mehnert]

Cannot reproduce this problem here with an Etch guest but a home-brewn 2.6.18 kernel. I can try to reproduce this later using the 2.6.18-Etch kernel but are you sure that you actually installed the latest additions?

[Martin Thielecke]

The additions installer says 2.1.0.

I've installed a Debian Etch (not etchnhalf!) with a netboot installer (6 MB image) without any additional packages (no GUI, no standard system, only the pure Debian system to keep it very small).

I changed the /etc/apt/sources.list to:

===

deb ​http://ftp.de.debian.org/debian/ etch main

deb ​http://security.debian.org/ etch/updates main contrib

deb ​http://www.backports.org/debian etch-backports main contrib non-free

===

And I executed:

aptitude -y update

aptitude -y dist-upgrade

aptitude install -y make linux-kernel-headers gcc linux-headers-uname -r

After that I've installed the Additions 2.1.0 and have the problems.

[Frank Mehnert]

I assume apache2 as well?

[Martin Thielecke]

Apache is 2.2.3-4+etch6

[Frank Mehnert]

Installed such an Etch system and I'm still not able to reproduce the problem (although using a Debian/Sid host). Is your host x86 or AMD64?

[Martin Thielecke]

My host and guest both are x86... Did you try a Netboot-Image?

[Yann]

please see also #2921

[Martin Burger]

I have the same issue on my virtual machine with Etch installed:

Linux etch08 2.6.24-etchnhalf.1-686 #1 SMP Fri Dec 26 04:10:16 UTC 2008 i686 GNU/Linux

Apache:

Apache/2.2.3 (Debian) PHP/4.4.4-8+etch6 configured

For me, EnableSendfile Off is sufficient as a workaround, I don't have to set EnableMMAP to Off.

[MrX1980]

Please try it with v3.0.0 again.

[Martin Burger]

If I comment EnableSendfile Off out, the failure occurs.

VirtualBox: 3.0.2 r49928

Can I provide more information that could be helpful?

[Frank Mehnert]

mburger, please could you attach a VBox.log file of such a VM session?

[Martin Burger]

I could reproduce the failure in VBox 3.0.4 r50677 and attached the requested log file.

[Frank Mehnert]

Please check VBox 3.0.10. Don't forget to update the guest additions. Reopen this ticket if necessary.

[Martin Burger]

I still have the same issue. Although I updated the guest additions. How can I double-check the version?

[Frank Mehnert]

In the guest, please execute

dmesg|grep vbox

and post the output here.

[Martin Burger]

# dmesg | grep vbox
vboxadd: Successfully loaded version 3.0.10 (interface 0x00010004)
vboxvfs: Successfully loaded version 3.0.10 (interface 0x00010004)

# uname -a
Linux xxx 2.6.24-etchnhalf.1-686 #1 SMP Sat Aug 15 16:51:49 UTC 2009 i686 GNU/Linux

[Frank Mehnert]

So that's still a problem. What is your test case, apache?

[Martin Burger]

Replying to frank:

So that's still a problem. What is your test case, apache?

Yes, Apache serves a static CSS file from a vboxsf.

[Supermighty]

I believe that I am suffering from this same issues.

I setup a Debian (Lenny 5.0) guest OS in virtualbox 3.0.12 on a Windows 7 pro host with ntfs (4k blocks). I setup a shared folder on the Win7 host that is mounted in the Debian guest. I have the apache web server setup on the guest and it points it's root folder to the mounted file share.

This is a great setup. I use host-only networking and I develop my php web app on the Win7 host and test using a web browser pointed at the Debian guest.

All was going well until I noticed that my css files were not loading the most up to date version. What I discovered was that apache served up corrupted css files when the css file was edited in the Win7 host OS.

When I edited the css file in the Win7 host OS apache served up files that had the black diamonds with question makes in them at the end of the file. I edited the css files with Eclipse, GVIM and notepad. The other thing was that the file served up did not have the most recent changes. If I used the host OS to edit the file and add a /* comment */ to the beginning or end of the file it did not show up. It doesn't matter where in the file the text is edited, when editing with host editors the black diamonds show up at the bottom of the file.

The problem corrected itself once I edited the css file in the Debian guest OS using VIM. I did a couple more test to see exactly what was going on. I copied the contents of a css file to 'test.html' so it was a css file but with an .html file extension and there was no issues. I could edit the test file in the host OS and have it served up by apache in the guest OS without any corruption. However when I changed the extension back to .css the problem resumed. I tested the other way too, creating an html file with .css as an extension and the corruption occurred, but not when the html file extension was reverted to .html.

I tried this with a couple different css files that I had and it seems that the problem only happens when the file size is more than ~2k. I could not replicate the problem on files that were ~120b but as soon as I added a bunch of css to the file to make it grow to more than ~4k the problem cropped up.

I have booted the Win7 host OS in restore mode and ran chkdsk while c:\ wasn't mounted and there were no returned errors. I have tried changing the css encoding from utf-8 to latin1.

To me it seems that css files are being corrupted when edited in the Win7 host OS if the css file is larger than ~2k on disk.

[Supermighty]

Update:

The guest OS is loaded and apache is running. I edit one of the css files in the host OS and save it. When I reload the css file in firefox in the host OS there are black diamonds with question marks at the bottom of the file.

I stop apache, unmount the vboxsf share, remount the share, and start apache. When I reload (f5) the file in Firefox the file is correct and as it should be. This leads me to believe that there is a problem in the way that vboxsf looks at how some files are changed or updated.

What can I do for the devs to help them track down this problem?

[Supermighty]

The solution that worked for me was suggested by a forum user (MarkCranness) was to turn off ​EnableSendfile in Apache. This changes the way that Apache reads a file before it serves it up when requested.

[Frank Mehnert]

Yes, this workaround is mentioned here in this ticket as well. We will try to fix this problem but not for 3.1.0 yet.

[Jesse Glick]

I seem to be having a similar problem with OpenSolaris 2009.06 guest in Ubuntu 9.10 host on VBox 3.0.12 using a freshly installed guest additions package. I mount a shared folder contain an Apache Ant 1.7.1 installation. Trying to run Ant fails:

$ /space/ant171/bin/ant -version mmap failed for CEN and END part of zip file Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/tools/ant/launch/Launcher

even though I can access its files otherwise:

$ jar tvf /space/ant171/lib/ant.jar | head -5

0 Fri Jun 27 05:03:52 EDT 2008 META-INF/

449 Fri Jun 27 05:03:50 EDT 2008 META-INF/MANIFEST.MF

0 Fri Jun 27 05:03:44 EDT 2008 org/ 0 Fri Jun 27 05:03:44 EDT 2008 org/apache/ 0 Fri Jun 27 05:03:50 EDT 2008 org/apache/tools/

Generally, running any other Java application based on JARs fails if the JARs are on a shared folder. If I 'cp -av' the app to a local disk it runs fine.

[Frank Mehnert]

Retry with 3.1.4 when released. Make sure to update the guest additions to 3.1.4 of course.

[Frank Mehnert]

Reopen if necessary.

[Jesse Glick]

Still easily reproducible in virtualbox-3.1 3.1.4-57640_Ubuntu_karmic running OpenSolaris 9.06 with SUNWvboxguest 3.1.4,REV=r57640.2010.02.12.20.26 (rebooted after updating new guest additions pkg).

jglick@jglick-solaris:~$ mount -v | fgrep /space space on /space type vboxfs read/write/setuid/devices/dev=4e00001 on Tue Feb 23 09:45:26 2010 jglick@jglick-solaris:~$ java -jar /space/tmp/sayhello/dist/sayhello.jar mmap failed for CEN and END part of zip file Exception in thread "main" java.util.zip.ZipException: error in opening zip file

at java.util.zip.ZipFile.open(Native Method) at java.util.zip.ZipFile.<init>(ZipFile.java:114) at java.util.jar.JarFile.<init>(JarFile.java:133) at java.util.jar.JarFile.<init>(JarFile.java:70)

jglick@jglick-solaris:~$ cp -v /space/tmp/sayhello/dist/sayhello.jar . /space/tmp/sayhello/dist/sayhello.jar' -> ./sayhello.jar' jglick@jglick-solaris:~$ java -jar sayhello.jar hello world jglick@jglick-solaris:~$ jar tvf /space/tmp/sayhello/dist/sayhello.jar

0 Tue Feb 23 09:48:08 EST 2010 META-INF/

207 Tue Feb 23 09:48:06 EST 2010 META-INF/MANIFEST.MF

0 Tue Feb 23 09:48:08 EST 2010 sayhello/

533 Tue Feb 23 09:48:08 EST 2010 sayhello/Main.class

jglick@jglick-solaris:~$

[Jesse Glick]

This may be easier to read:

Still easily reproducible in virtualbox-3.1 3.1.4-57640_Ubuntu_karmic running OpenSolaris 9.06 with SUNWvboxguest 3.1.4,REV=r57640.2010.02.12.20.26 (rebooted after updating new guest additions pkg).

jglick@jglick-solaris:~$ mount -v | fgrep /space
space on /space type vboxfs read/write/setuid/devices/dev=4e00001 on Tue Feb 23 09:45:26 2010
jglick@jglick-solaris:~$ java -jar /space/tmp/sayhello/dist/sayhello.jar
mmap failed for CEN and END part of zip file
Exception in thread "main" java.util.zip.ZipException: error in opening zip file
	at java.util.zip.ZipFile.open(Native Method)
	at java.util.zip.ZipFile.<init>(ZipFile.java:114)
	at java.util.jar.JarFile.<init>(JarFile.java:133)
	at java.util.jar.JarFile.<init>(JarFile.java:70)
jglick@jglick-solaris:~$ cp -v /space/tmp/sayhello/dist/sayhello.jar .
`/space/tmp/sayhello/dist/sayhello.jar' -> `./sayhello.jar'
jglick@jglick-solaris:~$ java -jar sayhello.jar
hello world
jglick@jglick-solaris:~$ jar tvf /space/tmp/sayhello/dist/sayhello.jar
     0 Tue Feb 23 09:48:08 EST 2010 META-INF/
   207 Tue Feb 23 09:48:06 EST 2010 META-INF/MANIFEST.MF
     0 Tue Feb 23 09:48:08 EST 2010 sayhello/
   533 Tue Feb 23 09:48:08 EST 2010 sayhello/Main.class
jglick@jglick-solaris:~$ 

[Frank Mehnert]

Moment, you are talking about a Solaris guest. Any positive feedback for Linux guests?

[Eric Herrera]

I have a similar problem, in that apache is delivering what appears to be a previous version of the file however truncated at the length of the current file.

Adding this to apache fixes the problem: EnableMMAP Off EnableSendfile Off

Here's the setup:

Host is: Ubuntu 9.10 Linux 2.6.31-19-generic #56-Ubuntu SMP Thu Jan 28 02:39:34 UTC 2010 x86_64 GNU/Linux VirtualBox 3.1.4 r57640

Guest is: Debian GNU/Linux 4.0 Linux 2.6.18-6-amd64 #1 SMP Thu Dec 25 21:08:54 UTC 2008 x86_64 GNU/Linux VirtualBox 3.1.4 Guest Additions for Linux MD5="2490b313eb8c0168a7d7648e12f5803d"

DocumentRoot is mounted vboxsf of host

[josephusmullally@gmail.com]

I recently came across what seems to be the same issue when trying to use mmap'd files on a vboxsf mounted share, on the (as of now) latest vbox version.

When files are opened read-only and used with mmap() it works fine, but it fails for files opened read-write and later used with mmap for writing.

Here is some code to reproduce:

// mmap_test.c
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <error.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/mman.h>

int main(int argc, char *argv[])
{
    char *ro_mem, *rw_mem;      // mmapped IO
    int ro_fd, rw_fd;
    off_t len_ro, len_rw;

    // read test - works OK on vboxsf
    if( (ro_fd = open("ro_file", O_RDONLY)) == -1)
        error(-1, errno, "Error opening 'ro_file'");
    len_ro = lseek(ro_fd, 0L, SEEK_END);
    if((ro_mem = (char *) mmap(NULL, len_ro, PROT_READ, MAP_SHARED, ro_fd, 0)) == MAP_FAILED)
        error(-1, errno, "Error memory-mapping ro_file");
    printf("ro_mem contents:   %s", ro_mem);
    munmap(ro_mem, len_ro); close(ro_fd);

    // write test - fails on vboxsf
    if( (rw_fd = open("rw_file", O_RDWR)) == -1)
        error(-1, errno, "Error opening 'rw_file'");
    len_rw = lseek(rw_fd, 0L, SEEK_END);
    if((rw_mem = (char *) mmap(NULL, len_ro, PROT_READ|PROT_WRITE, MAP_SHARED, rw_fd, 0)) == MAP_FAILED)
        error(-1, errno, "Error memory-mapping rw_file");
    rw_mem[5] = 'X';
    munmap(rw_mem, len_rw); close(rw_fd);

    // Changing the rw case to MAP_PRIVATE will succeed, but changes 
    // aren't propogated to 'rw_file', as expected.
    // Removing PROT_WRITE still fails.
    // --> dies on open(O_RDWR),mmap(MAP_SHARED) pair

    return 0;
}

#!/bin/sh
# test.sh
gcc -o mmap_test mmap_test.c
echo "Sample text for readonly file" > ro_file
echo "Write test text" > rw_file
./mmap_test
cat rw_file

# Output on a regular ext3fs dir:
#    ro_mem contents:   Sample text for readonly file
#    WriteXtest text

# Output on a vboxsf dir:
#    ro_mem contents:   Sample text for readonly file
#    ./mmap_test: Error memory-mapping rw_file: Invalid argument
#    Write test text

versions:
Host:   Windows 7, Virtualbox 3.1.6 r59338

Guest:  Ubuntu 9.10 Karmic - 2.6.31-20-generic-pae
        (Using vbox guest additions from host supplied ISO)
    
        vboxguest: major 0, IRQ 9, I/O port d040, MMIO at 00000000f0400000 (size 0x400000)
        vboxguest: Successfully loaded version 3.1.6 (interface 0x00010004)
        vboxvfs: Successfully loaded version 3.1.6 (interface 0x00010004)

Hope that helps

[mesnakh]

Host: Windows 7 64-bit running Virtual Box 3.1.8, r61349 Guest: Up to date Arch Linux: Linux perelman 2.6.33-ARCH #1 SMP PREEMPT Sun May 2 10:40:03 CEST 2010 x86_64 Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz GenuineIntel GNU/Linux Guest additions for 3.1.8 have been installed and are running.

To reproduce the problem:

1. Run rtorrent 0.8.6 with a vboxsf mount as its download directory
2. Try to download any torrent
3. Observe every download failing with "Storage error: [File chunk write error: Invalid argument.]"

rtorrent uses mmap for this.

[Bryan Drewery]

I'm having this exact problem in 3.1.8 (OpenSolaris 134 Host) in a Linux guest. I'm unable to try newer versions at this time. Reading a file with mmap() works fine while writing it does not.

[zimbatm]

Same problem with VirtualBox 3.2.12, Host: OSX 10.6.5, Guest: ubuntu-lucid-32

[Alexey Fyodorov]

Same problem with VirtualBox 4.0.4, Host: Windows 7 x64, Guest: Oracle Solaris 11 Express 2010.11

[Jesse Glick]

No longer having any problems using Ubuntu host and XP guest (w/ 4.0.4 guest additions); ZIP file access working as expected.

[carstep]

Same issue with 4.0.8 on W7 ultimate and debian(lenny) guest with apache2 with JavaScript files EnableSendfile Off didn't help only sync; echo 3 > /proc/sys/vm/drop_caches, but running this every time I develop further is inconvenient

18. Sandor

[carstep]

update:

using EnableMMAP Off helped

[Frank Mehnert]

carstep, do you read files from the guest or do you write files to the guest?

[carstep]

I write files to the quest

[David-Sarah Hopwood]

The mmap problem may be the cause of ​http://bugs.darcs.net/issue2091 , where darcs fails to write to a repository stored in a vbox shared folder.

[David-Sarah Hopwood]

Replying to davidsarah2:

The mmap problem may be the cause of ​http://bugs.darcs.net/issue2091 , where darcs fails to write to a repository stored in a vbox shared folder.

Note that this problem occurs still in VirtualBox 3.2.8_OSE r64453. In my case, the host is Ubuntu 10.10 and the shared folder is accessed from a Windows XP SP3 guest, although the original reporter of the darcs bug also had the problem when accessing a shared folder from Linux.

[SimonSimCity]

I can confirm this issue - not using Apache but using nginx. I don't know an option of nginx like EnableMMAP what you've written concerning Apache.

I've moved back to mount my shared-folder by samba to work.

To work arround this problem you can clear the pagecache of linux every second - like this:

watch -n 1 `sync; echo 1 > /proc/sys/vm/drop_caches`

Source: ​http://www.linuxinsight.com/proc_sys_vm_drop_caches.html

[SimonSimCity]

Sorry for this double-post .. but I'm not able to edit my last information. Here's another link where they explain a simular problem using more text :) ​http://serverfault.com/questions/30240/disable-all-disk-caching-for-apache2-on-linux

[Jesse Glick]

Using virtualbox-4.1.2-dfsg-1ubuntu1 on Ubuntu Oneiric with an XP guest (4.0.4 guest additions) I continue to have this problem sometimes, but not always; certain JAR files cannot be accessed from the guest, giving a ZIPException. Dropping the page caches from the host seems to make no difference. Using guest additions 4.1.2 it seems to be working better. Perhaps related to ticket #9276, mentioned in the changelog for this release?

[laoshanlong]

Also getting this issue with Virtualbox 4.1.6. Using a Windows 7 x64 host and an Ubuntu Server guest. Fresh install from today. Attempting to use rtorrent to download to a vboxsf mount results in Storage error: [File chunk write error: Invalid argument.], an error that occurs due to an issue with mmap.

[rhcoe]

I'm experiencing a similar issue on VirtualBox 4.1.6, first experienced on 4.1.4. java apparently uses mmap to read zip'd jar files, and this fails for files larger than 2129615 (0x207ecf) bytes.

The next largest jar is 2213055 (0x21c4bf) and cannot be read.

I'm running a win xp guest on a linux host.

e: is a mounted shared directory from the linux host.

jar tvf e:\some\file.jar
java.util.zip.ZipException: error in opening zip file
    at java.util.zip.ZipFile.open(Native Method)
    at java.util.zip.ZipFile.<init>(ZipFile.java:127)
    at java.util.zip.ZipFile.<init>(ZipFile.java:88)
    at sun.tools.jar.Main.list(Main.java:979)
    at sun.tools.jar.Main.run(Main.java:224)
    at sun.tools.jar.Main.main(Main.java:1149)

[LemADEC]

Issue reproduced under Apache when updating .css files.
Disabling caches and MMAP didn't help.
Disabling also Sendfile fixed the issue.

Version used are:
Host = Windows 7 SP1 32 US
Guest = Entreprise Linux 2.6.32-279.11.1.el6.x86_64 + httpd-2.2.15-15.el6
using VirtualBox 4.2.0 r80737 + vboxguest/vboxsf 4.2.0

[Bartzy]

Any updates on this issue?

Thanks!

[eKIK]

Issue is still occurring with VirtualBox 4.2.16, Ubuntu 13.10 host and Ubuntu 12.04 guest.

A workaround for nginx is to set "sendfile off;" (​http://nginx.org/en/docs/http/ngx_http_core_module.html#sendfile).

[eugeneware]

This is still an issue in the current VirtualBox version 4.3.6 with Ubuntu 12 and 13.

When using mmap operations on a vboxfs mapped folder it fails.

I was trying to use leveldb and lmdb operations on vboxfs mount, and couldn't open the files. Using an nfs share works however.

Here's some code that replicates the problem:

​https://gist.github.com/9045290

If you compile this mmap-test, then running:

$ mmap-test /file/on/vboxfs/partition

fails with an invalid argument error

However, running it with a file on an nfs or ext4 partition works fine.

This is messing up local development workflows for anything that uses mmap (ie. a lot of software).

Please fix!

[eugeneware]

Just checking to see if anyone is currently working on this? Frank? Any luck replicating?

[eugeneware]

Hi Frank. I've just tested the latest version of VirtualBox (4.3.8) and the problem still remains?

Any update?

[Frank Mehnert]

Sorry, no update. Currently we have other priorities. If mmap-access to shared folders does not work, try to prevent it. For instance, Apache can be forced to not access files this way. If you are capable of Linux programming you might also want to check the source code. Patches help to fix such bugs faster.