Tuesday, January 21, 2014

VMware Player 6.0.1 on Ubuntu 14.04 alpha - Modules compile error

Updating Ubuntu 14.04 (alpha / development) with latest 3.13.0-x kernels will require recompiling the modules for VMware Player 6.0.1 which will fail as per below. The solution is based on Garrett Skjelstad's "VMware modules & kernel 3.13" blog post.
$ sudo lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu Trusty Tahr (development branch)
Release: 14.04
Codename: trusty

$ uname -a
Linux VOSTRO 3.13.0-4-generic #19-Ubuntu SMP Thu Jan 16 18:10:11 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

$ vmplayer --version
VMware Player 6.0.1 build-1379776

  CC [M]  /tmp/modconfig-7ykUGm/vmnet-only/filter.o
/tmp/modconfig-7ykUGm/vmnet-only/filter.c:206:1: error: conflicting types for ‘VNetFilterHookFn’
 VNetFilterHookFn(unsigned int hooknum,                 // IN:
/tmp/modconfig-7ykUGm/vmnet-only/filter.c:64:18: note: previous declaration of ‘VNetFilterHookFn’ was here
 static nf_hookfn VNetFilterHookFn;
/tmp/modconfig-7ykUGm/vmnet-only/filter.c:64:18: warning: ‘VNetFilterHookFn’ used but never defined [enabled by default]
/tmp/modconfig-7ykUGm/vmnet-only/filter.c:206:1: warning: ‘VNetFilterHookFn’ defined but not used [-Wunused-function]
 VNetFilterHookFn(unsigned int hooknum,                 // IN:
make[2]: *** [/tmp/modconfig-7ykUGm/vmnet-only/filter.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [_module_/tmp/modconfig-7ykUGm/vmnet-only] Error 2
make[1]: Leaving directory `/usr/src/linux-headers-3.13.0-4-generic'
make: *** [vmnet.ko] Error 2
make: Leaving directory `/tmp/modconfig-7ykUGm/vmnet-only'
Failed to build vmnet.  Failed to execute the build command.
Starting VMware services:
   Virtual machine monitor                                             done
   Virtual machine communication interface                             done
   VM communication interface socket family                            done
   Blocking file system                                                done
   Virtual ethernet                                                   failed
   VMware Authentication Daemon                                        done

To fix this we will need to apply this patch to filter.c in VMware Player module sources. Start with saving the diff below (download):
> #else
> VNetFilterHookFn(const struct nf_hook_ops *ops,        // IN:
> #endif
<    transmit = (hooknum == VMW_NF_INET_POST_ROUTING);
>       transmit = (hooknum == VMW_NF_INET_POST_ROUTING);
>    #else
>       transmit = (ops->hooknum == VMW_NF_INET_POST_ROUTING);
>    #endif

Or download directly to the system:
$ wget https://sites.google.com/site/dandar3blogspotcom/vmware-player-601-on-ubuntu-1404-alpha-filter.c.diff \
       -O /tmp/filter.c.diff --no-check-certificate

Next will apply the patch to filter.c in vmnet.tar (extract, patch, update archive):
$ sudo -E -s

# cd /usr/lib/vmware/modules/source/ 

# cp vmnet.tar vmnet.tar.original

# tar xvf vmnet.tar vmnet-only/filter.c 

# patch vmnet-only/filter.c < /tmp/filter.c.diff 
patching file vmnet-only/filter.c

# tar -uvf vmnet.tar vmnet-only/filter.c 

# rm -rf vmnet-only/
And now you're ready to restart VMplayer, which will ask you to re-compile VMware Player modules, successfully this time.