FastSwap
FastSwap copied to clipboard
Dynamic and Transparent Memory Sharing for Accelerating Big Data Analytics Workloads in Virtualized Cloud
trafficstars
FastSwap
Overview
Memory resource continues to be the biggest demand for running big data analytics jobs. Guest swapping is an OS virtual memory facility to keep big data jobs running in the presence of high memory pressure on guest virtual machines (VM). Thus, the performance of guest swapping may affect the latency of big data applications in virtualized Clouds. Here, we present FastSwap, a high performance shared-memory swapper. The design of FastSwap makes a number of original contributions.
- First, FastSwap improves guest swapping performance by leveraging idle host memory and redirecting the guess swapping traffic to the host-guest compressed shared memory swap area.
- Second, FastSwap develops a compressed swap page table as an efficient index structure to provide high utilization of shared memory swap area through supporting multi-granularity of compression factors.
- Third, FastSwap provides hybrid swap-out and two-level proactive swap-in to further improve the performance of shared memory swapping.
Getting Started
###On Host
- Build QEMU on the host
- apply the patch file under folder
FastSwap-QEMU-Patchtoqemu-2.0.0 - compile
qemu-2.0.0 - install
qemu-2.0.0
- apply the patch file under folder
- Create shared memory on the host
- go to folder
Ivshmem-Server - run
./ivshmem.sh(this will create a 16GB shared memory on host) - run
chmod 777 /tmp/ivshmem_socket
- go to folder
- Create a VM
virsh create fastswap.xml(the xml file is included in package, you may need to change some fields in this file)- set swapfile as 16GB when installing operating system
###On VM
- Build Linux kernel
- start VM by executing command
virsh create fastswap.xml - download Linux kernel 3.14.4 from
https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.14.4.tar.gz - apply Linux kernel patch under folder
FastSwap-Kernel-Patch - recompile and install kernel 3.14.4
- restart VM and boot with kernel 3.14.4
- start VM by executing command
- Start FastSwap
- copy folder
GuestOS-Kernel-Moduleto GuestOS - execute
./run - execute
./run(execute this twice to fix some memory initialization issue)
- copy folder
Now, you're all set.