Talk given at Apache Roadshow, FOSS Backstage, Berlin, June 2018
Apache CloudStack is open source software designed to deploy and manage large networks of virtual machines, as a highly available, highly scalable Infrastructure as a Service (IaaS) cloud computing platform. This talk will give an introduction to the technology, its history and its architecture. It will look common use-cases (and some real production deployments) that are seen across both public and private cloud infrastructures and where CloudStack can be completed by other open source technologies.
The talk will also compare and contrast Apache Cloudstack with other IaaS platforms and why he thinks that the technology, combined with the Apache governance model will see CloudStack become the de-facto open source cloud platform. He will run a live demo of the software and talk about ways that people can get involved in the Apache CloudStack project.
Fact vs. Fiction: Autodetecting Hallucinations in LLMs
Building clouds with apache cloudstack apache roadshow 2018
1. Building Clouds with Apache
CloudStack
Apache Roadshow, Berlin
Giles Sirett
A n i n t r o d u c t i o n t o
2. C l i c k t o e d i t
@CloudStack @GilesSirett
A b o u t m e
• PMC member & Committer -
Apache CloudStack
• Chair –CloudStack European
user group
• CEO - ShapeBlue
• Declaration: I’m a business
guy
• Also: Physicist, ex-
programmer, husband, runner,
Dad, rugby coach, proud Brit
AND European
3. C l i c k t o e d i t
@CloudStack @GilesSirett
W h a t i s C l o u d S t a c k ?
Apache CloudStack is a scalable, multi-
tenant, open-source, purpose-built,
cloud orchestration platform for
delivering turnkey Infrastructure-as-a-
Service clouds
4. C l i c k t o e d i t
@CloudStack @GilesSirett
H o w t o b u i l d a n I a a S c l o u d
compute StorageNetworking
Hypervisor
CMP - Orchestration layer
orchestration API
eCommerce Platform
Choice of Hypervisor (KVM, VMWare,
Xen, hyper-V, OVM, XCP-ng)
Apache CloudStack
Ecommerce platform
CloudStack API
CaaS Management
Networking compute Storage
PaaS Developer tooling
Multi-cloud
management
Cloud Monkey
5. C l i c k t o e d i t
@CloudStack @GilesSirett
K e y c l o u d S t a c k f e a t u r e s
• Broad & deep Hypervisor support
• XenServer, KVM, VMware, OracleVM, Hyper-V, XCP-ng
• Enterprise grade tenant Virtual Networking model
• Scalable architecture
• Support thousands of hosts and virtual machine guest
• Largest known production cloud 35k+ physical hosts
• High availability
• Configurations that provide automatic failover for virtual
machines
• Choice of interfaces
• Web UI, command line, REST-based API
6. C l i c k t o e d i t
@CloudStack @GilesSirett
C l o u d S t a c k h i s t o r y
• 2008 – Vmops launches (rebrands Cloud.com)
• April 2010 – Cloud.com Releases CloudStack under GPLv3
• July 2011 - Citrix acquires CloudStack
• April 2012 - CloudStack donated to Apache Software Foundation
• November 2012 – First Apache CloudStack Release
• March 2013 - CloudStack graduates from ASF Incubator
• 2014 – Majority of committers are not from Citrix
• 2016 – Citrix leave the project
• 2014-2018 Lots of releases, adoption continues to grow
7. C l i c k t o e d i t
@CloudStack @GilesSirett
C l o u d S t a c k t o d a y
• 3-4 releases per year
• Widespread production deployment
Public cloud providers
Private cloud use-cases
• LTS releases
• Diverse user driven developer community…..
8. C l i c k t o e d i t
@CloudStack @GilesSirett
C l o u d S t a c k t o d a y
• C.200 project committers
• Diverse PMC
• Last 4 weeks (to 11 June)
• 400 mailing list msgs
• 60 mailing list contributors
• 34 merged PRs, 13 authors
• 1600 package downloads
• Lots of meetups &
events….
9. C l i c k t o e d i t
@CloudStack @GilesSirett
A p a c h e C l o u d S t a c k C o m m u n i t y
200+ active contributors
7o+ organisations
Now 72 86 committers
10. C l i c k t o e d i t
@CloudStack @GilesSirett
C l o u d S t a c k C o l l a b o r a t i o n C o n f e r e n c e s
http://Cloudstackcollab.org
11. C l i c k t o e d i t
@CloudStack @GilesSirett
C l o u d S t a c k u s e - c a s e s
Private Cloud Hybrid Cloud Public Cloud
12. C l i c k t o e d i t
@CloudStack @GilesSirett
C l o u d s t a c k “ k n o wn u s e r s ”
13. C l i c k t o e d i t
@CloudStack @GilesSirett
C l o u d s t a c k * k n o wn u s e r s ”
19
14. C l i c k t o e d i t
@CloudStack @GilesSirett
C l o u d S t a c k * k n o wn * u s e r s
15. C l i c k t o e d i t
@CloudStack @GilesSirett
W h a t c a n y o u d o wi t h C l o u d S t a c k ?
• Self service of all resources – compute, storage and networking –
with no requirements for highly skilled technical staff.
• Automation of all provisioning and management through API.
• E.g.
• Create Virtual Machines from templates or ISOs
• All Virtual Machine lifecycle actions: start/stop/delete/storage/networking
• Manage storage volumes
• Create Isolated, Shared and Multi-Tiered Networks
• Manage firewall and port forwarding rules
• Manage Network Services such as Load Balancing, Static and Source NAT,
VPNs, Global Load Balancing and Autoscaling
17. C l i c k t o e d i t
@CloudStack @GilesSirett
A P I e x a m p l e s
Examples of Cloudstack Rest API:
• List VMs for all users:
http://10.2.3.183:8096/client/api?command=listVirtualMachines&listall=true
• List all my templates:
http://10.2.3.183:8096/client/api?command=listTemplates&templatefilter=all
• Create new network "RootNet2"
– http://10.2.3.183:8096/client/api?command=createNetwork&zoneId=ea2edb5f-2fbd-4390-a731-
b4e252d2f5c7&name=RootNet2&displayText=RootNet2&networkOfferingId=d323caa4-976b-4cd3-b57c-
58864b842a40&domainid=d30d22bf-6afd-11e8-aa6b-06227e0108af&account=admin
• Create a new VM attached to RootNet1
– http://10.2.3.183:8096/client/api?command=deployVirtualMachine&response=json&zoneid=ea2edb5f-
2fbd-4390-a731-b4e252d2f5c7&templateid=d3121fc3-6afd-11e8-aa6b-
06227e0108af&hypervisor=VMware&serviceofferingid=89e3afd9-3ed6-4975-a9a9-
58953f94a9f2&iptonetworklist%5B0%5D.networkid=af048a31-f140-4907-b2ac-
85ae67767ff9&displayname=RootVM2&name=RootVM2&keyboard=uk&domainid=d30d22bf-6afd-11e8-
aa6b-06227e0108af&account=admin
Note: for simplicity, these commands are being run over an unauthenticated port (8096) - this is not recommended
for production use and would usually be disabled
18. C l i c k t o e d i t
@CloudStack @GilesSirett
C l o u d m o n k e y
19. C l i c k t o e d i t
@CloudStack @GilesSirett
T h e p e r c e p t i o n p r o b l e m
“CloudStack –
Never heard of
it!”
Architect at unnamed
company, 2016, that
now runs a 800 host
production Cloudstack
production environment
20. C l i c k t o e d i t
@CloudStack @GilesSirett
I n 2 0 11
21. C l i c k t o e d i t
@CloudStack @GilesSirett
Wrong
place, wrong
time
falscher Ort
falsche Zeit
22. C l i c k t o e d i t
@CloudStack @GilesSirett
We L O V E t h e A S F, b u t … .
The mission of the Apache Software
Foundation (ASF) is to provide software for
the public good. We do this by providing
services and support for many like-minded
software project communities of individuals
who choose to join the ASF.
23. C l i c k t o e d i t
@CloudStack @GilesSirett
24. C l i c k t o e d i t
@CloudStack @GilesSirett
O p e n s t a c k i s c o m p l e x
25. C l i c k t o e d i t
@CloudStack @GilesSirett
C l o u d S t a c k i s s i m p l e
Availability and Security
Servers Network Storage
Virtualization Layer
Service Management (Billing, Metering, Accounts, etc.)
Resource Management
Servers Storage Network
Dynamic Workload Management
Backup LB HA Monitoring
User Interface Developer API
Image Libraries
Application Catalog
Custom Templates
Operating System ISOs
IntegrationAPI
OperationalIntegration
(OSS/BSS,Monitoring,IdentityManagement,Etc)
Administrator End User Console
26. C l i c k t o e d i t
@CloudStack @GilesSirett
W h y C l o u d S t a c k ?
• Integrated end-to-end IaaS product
• Proven at scale, widespread adoption
• Rapid time to value
• Low implementation & operational
costs
• Truly multi-tenant
• Focussed, user led, development
community
• Narrow scope / easy integration
• #CloudStackWorks
27. C l i c k t o e d i t
@CloudStack @GilesSirett
A n d i n 2 0 1 7
28. C l i c k t o e d i t
@CloudStack @GilesSirett
A p a c h e C l o u d S t a c k – a n o p e n f l e x i b l e p l a t f o r m
XenServer VMware OVM KVM UCS
Compute
Local Disk iSCSI
Fibre
Channel
NFS
Compute primary storage
Bare metalHyper-V
NFS Swift S3
User accessible secondary storage
Ceph
29. C l i c k t o e d i t
@CloudStack @GilesSirett
C l o u d S t a c k n e t wo r k i n g – 2 m o d e l s
Services
Isolation
Basic – L3
Firewall
LB GSLB
Network
VPN
Advanced – L2
Routing DHCP DNS
30. C l i c k t o e d i t
@CloudStack @GilesSirett
C l o u d S t a c k - S c a l a b l e c o n s t r u c t s
Region - Europe
Zone –
London1
Zone- Amsterdam3
Pod PodCluster -
Vmware
Cluster -
KVM
Region -
USA
Cluster –
OVM
Host
Host
31. C l i c k t o e d i t
@CloudStack @GilesSirett
D e p l o y m e n t m o d e l
Internet
facing
servicesUsers
Management
server(s)
MySQL
DBs
Hosts
Primary
storage
Hosts
Primary
storage
Hosts
Primary
storage
Pod1 Pod2 Pod3
Secondary
Storage
32. C l i c k t o e d i t
@CloudStack @GilesSirett
C l o u d S t a c k c o n t a i n e r S e r v i c e
• Seamless Container as a Service offering
in IaaS environment
• Built with Kubernetes & CloudStack
• Deploy cloud-native apps into clusters
• No changes to product catalogue or
billing process
33. C l i c k t o e d i t
@CloudStack @GilesSirett
G e t t i n g s t a r t e d wi t h A p a c h e C l o u d s t a c k
Recipe 1
completely ignore design best-
practice
• Ingredients: A hypervisor
host, some NFS storage,
CloudStack Management
server, Cloudstack
MYSQL server
34. C l i c k t o e d i t
@CloudStack @GilesSirett
G e t t i n g s t a r t e d wi t h A p a c h e C l o u d s t a c k
Method
1. Read the recipe & understand
your ingredients
http://docs.cloudstack.apache.
org/en/latest/
2. Read on Cloudstack
networking
3. Does CloudStack
support?……
1. It’s the hypervisor stupid
4. Do not try to do this in AWS –
you need hypervisor hosts
35. C l i c k t o e d i t
@CloudStack @GilesSirett
G e t t i n g s t a r t e d wi t h A p a c h e C l o u d s t a c k
Method
5. Install MYSQL
6. Install Cloudstack
7. Start cloud XXX
8. Follow the GUI
36. C l i c k t o e d i t
@CloudStack @GilesSirett
M y f i r s t C l o u d S t a c k b u i l d
Host 1
• Centos
• MySQL DB for
Cloudstack
• Cloudstack
Management
server
• NFS storage
Host 2
• Hypervisor
host of your
choice (as long
as its KVM or
Xen)
Switch with VLAN support
37. C l i c k t o e d i t
@CloudStack @GilesSirett
L e a r n i n g C l o u d s t a c k f e a t u r e s – u s e s o m e b o d y e l s e s
• Interoute VDC
• Full stack IaaS offering
based on Cloudstack
• Includes hybrid “VDC
Edge”
• Massive growth – now
17 global zones
38. C l i c k t o e d i t
@CloudStack @GilesSirett
L e a r n i n g C l o u d S t a c k f e a t u r e s - u s e s o m e b o d y e l s e s
• Swiss based cloud
provider, based on
CloudStack
• “made for developers”
• Abstracted a number
of CloudStack featurs
39. C l i c k t o e d i t
@CloudStack @GilesSirett
Recipe 2
• More realistic storage
Primary
Secondary
• Add at lest one more host
Test deployment
Test HA
• Consider pod, clusters
• Introduce 2nd hypervisor
Understand different hypervisor
communication techniques
40. C l i c k t o e d i t
@CloudStack @GilesSirett
Recipe 2
• Build a management farm
Spilt MYSQL, use master/slave
Redundant pair of managemenr
servers
• Split Storage, guest,
public and management
networks
41. C l i c k t o e d i t
@CloudStack @GilesSirett
D e p l o y m e n t m o d e l
Internet
facing
servicesUsers
Management
server(s)
MySQL
DBs
Hosts
Primary
storage
Hosts
Primary
storage
Hosts
Primary
storage
Pod1 Pod2 Pod3
Secondary
Storage
42. C l i c k t o e d i t
@CloudStack @GilesSirett
D e s i g n c o n s i d e r a t i o n s – t h e r e a l wo r l d
• Analyse your workloads
• Capacity planning:
Storage
CPU/memory
• Version of CloudStack (go LTS)
• Choose networking model
• Management farm
Resilience – multiple management servers & DB servers
• Hypervisor choice
Good for your workloads
Compatible with your hardware?
Some picture about
restaurant quality
43. C l i c k t o e d i t
@CloudStack @GilesSirett
D e s i g n c o n s i d e r a t i o n s – t h e r e a l wo r l d
• Network design: isolate management, guest, public &
storage traffic
• Scale
scale point – usually by pod
• Plan templates
• Plan service offerings
• Plan disk offerings
• Tagging model ?
• Collecting & analysis of your usage data
44. C l i c k t o e d i t
@CloudStack @GilesSirett
W h e r e t o g o n e x t
• Documentation http://docs.cloudstack.apache.org/en/latest/
• Mailing lists
https://cloudstack.apache.org/mailing-lists.html
Start with users list
• Events & meetups
CloudStack European User Group
https://www.linkedin.com/groups/4294158
CloudStack Collaboration Conferences
http://cloudstackcollab.org
Specifically in Germany
https://www.meetup.com/german-CloudStack-user-group/
https://listi.jpberlin.de/mailman/listinfo/german-cloudstack-usergroup
45. C l i c k t o e d i t
@CloudStack @GilesSirett
Giles.Sirett@ShapeBlue.com
Giles@apache.org
@GilesSirett @cloudstack
www.cloudstack.org
#CloudStackWorks