As a solutions architect, I constantly seek innovative ways to overcome resource limitations while delivering impactful results.
One particularly challenging yet enlightening experience came during my MSc. Data Science dissertation at London South Bank University. My project, titled Network Anomaly Detection in SD-WAN using Machine Learning, required robust computational resources to generate data and simulate complex network environments.
Unfortunately, my local system—a standard gaming laptop—fell woefully short of these demands.
The university’s Systems Lab, while normally an excellent resource, was temporarily unavailable due to maintenance. Purchasing a high-performance workstation was financially impractical. Faced with these constraints, I embarked on a journey to creatively leverage available resources.
Finding the Solution
My research led me to an insightful video series by David Bombal, which highlighted the potential of combining local and cloud resources for running GNS3 simulations. This approach aligned perfectly with my project needs. However, one significant hurdle remained: the appliances I required, particularly Cisco SD-WAN, necessitated nested virtualization—a feature not supported by all cloud providers.
To address this, I devised a hybrid solution combining the computational power of:
- My Local Laptop: Acting as the primary control hub for running GNS3 and managing simulations.
- DigitalOcean Cloud Instance: Chosen for its support for nested virtualization, critical for running Cisco SD-WAN appliances effectively.
- A Dedicated Server: Generously provided by a friend, offering additional compute and memory capacity.
To interconnect these disparate resources, I implemented a VPN mesh using OpenVPN. This setup created a seamless network environment, enabling appliances hosted on different GNS3 VMs to communicate as though they were on the same local network. The architecture’s core components included VPN connectivity, distributed GNS3 instances, and optimized resource allocation based on the workload demands.
Pulling everything together, I managed to create a setup using local, cloud, and remote systems that supported my project needs.
Have you faced similar challenges? If so, I’d love to hear your solutions or explore how I could help you tackle them.