Hybrid and Multi-Cloud Strategies in Australia

Hybrid and multi-cloud strategies are becoming table stakes for forward-looking organisations in Australia — and for good reason. Across industries, cloud strategies have been put in place to support business objectives. As cloud adoption deepens, leaders are looking for smarter, more resilient IT.

Cloud platforms offer scale, global reach, smarter cost control, and resilience when things go wrong.

Relying on just one cloud provider can affect your cyber resilience and service availability

Digital change is moving fast, compliance demands are getting tougher, and market conditions are shifting constantly. Leaders need IT that bends, not breaks.

Hybrid and multi-cloud aren’t simply tech buzzwords. They’re the way Australian organisations stay in control, meet compliance, and build resilience.

What is Hybrid Cloud? What is Multi-Cloud?

Hybrid cloud

Hybrid cloud blends private IT infrastructure with public cloud services. Sensitive or regulated data stays under your control in a private environment, while public cloud gives you the flexibility to scale when demand spikes.

Think of it as keeping your most valuable assets in a safe at home, while renting extra storage space when you need room for the less sensitive stuff. It’s about balancing control with flexibility, using the right environment for the right job.

Multi-cloud

Multi-cloud takes it a step further. Instead of committing everything to one provider, businesses spread workloads across platforms like AWS, Azure, or Google Cloud. This avoids lock-in, reduces risk, and gives organisations the freedom to use the best tool for each task.

It’s a bit like diversifying your investments. You wouldn’t put your entire portfolio into one stock. Multi-cloud works the same way — spreading risk, increasing resilience, and giving you more leverage when the market shifts.

Why Australian Businesses Are Going Hybrid and Multi-Cloud

Choosing the Best Services

No single provider does everything perfectly. Amazon Web Services are well-known for innovative solutions and cutting edge services. Microsoft 365, Dynamics 365, Power Platform and Azure provides an established ecosystem of building blocks that interoperate almost seamlessly.

Cloud management tools have been improving. Now there are solid, dependable tools for managing your hybrid cloud / multi-cloud platforms. Azure Arc is gaining a lot of traction.

Compliance and sovereignty.

Many organisations keep sensitive data secure in their own infrastructure while using a public cloud for extra capacity.

This is carefully considered in Australia with many high-profile data privacy breaches reported lately. Regulations demand that patient records, financial data, and other sensitive information remain onshore under strict control — a clear case of data sovereignty shaping how organisations use the cloud.

Hybrid cloud makes compliance more straightforward than relying on public cloud infrastructure: keep the sensitive data local, let other workloads benefit from global scalability. Compliance, tick. Agility maintained.

Resilience and Return on Investment.

By blending in-house systems with cloud platforms, organisations create stronger safety nets.

The payoff is real. Industry reports, including the 2023 Intel analysis, show that enterprises applying disciplined FinOps practices — cost visibility, optimisation, and governance — have improved cloud ROI by as much as 30% in a single year. These results highlight the tangible business benefits of hybrid and multi-cloud strategies when paired with strong financial operations.

But pause for a moment — what would just one hour of downtime cost your business in sales, trust, and reputation?

Reducing Risk with Hybrid and Multi-Cloud

Escaping vendor lock-in.

One of the biggest risks of going all-in with one provider is getting stuck. If costs rise or service dips, you’ve got little choice.

Multi-cloud changes that. By spreading workloads, businesses gain leverage, independence, and flexibility. Ask yourself: what’s your plan if your only provider suddenly changes the rules?

Stronger disaster recovery.

Resilience isn’t a buzzword. It’s survival. Hybrid cloud setups let critical systems mirror each other between private and public environments — if one fails, the other steps in.

Multi-cloud goes further, shifting workloads across providers and regions. For banks, healthcare, and retail — where every minute counts — this isn’t optional. It’s a business imperative.

Practical Advice for Building a Hybrid Strategy

Moving to hybrid or multi-cloud isn’t just about adding a second provider. It’s about designing smarter. Start here:

  • Map workloads. Keep regulated data close to home. Run seasonal or high-volume apps in the public cloud.
  • Stay cloud-agnostic. Use tools like Kubernetes or Terraform to stay portable.
  • Build in security. Encrypt data, apply Zero Trust, check compliance often.
  • Simplify operations. Unified dashboards and automation prevent complexity from spiralling.
  • Track costs. Apply FinOps to right-size resources and monitor usage continuously.
  • Invest in people. Upskill IT teams or bring in experienced partners. The best plan fails without the right people behind it.

Think of it like building a winning sports team. Technology is the stadium, but it’s people who win the game.

Why Now Is the Time?

Hybrid and multi-cloud aren’t passing fads. They’re the new building blocks for businesses that want to stay competitive, compliant, and resilient. This shift reflects a wider digital transformation across Australia, where cloud-first has become the norm.

The bottom line: organisations that adopt hybrid strategies today will be better prepared for tomorrow — whether that’s scaling fast, meeting new compliance rules, or surviving an unexpected outage.

So here’s the question: will your IT bend when the pressure comes, or will it break? For organisations exploring hybrid cloud Australia and multi-cloud Australia solutions, the time to act is now.

Hybrid and multi-cloud are your multi-pronged digital foundation — flexible, reliable, and ready for whatever comes next.

What’s Next for Your Cloud Strategy?

If your business is still tied to a single provider, now is the time to rethink. Start small, map your workloads, and explore what hybrid and multi-cloud could unlock for you.The question isn’t if you’ll need it. The question is how soon you can get it right.

Resources

The Global AI Investment Race: US, China, UK and Australia in 2025 and Beyond

As 2025 unfolds, the global race to lead in Artificial Intelligence is marked by extraordinary new levels of investment, infrastructure buildout, and competition for talent. The US, China, UK, and Australia each approach this opportunity and challenge from distinct positions—shaped by capital flows, infrastructure ambitions, market scale, and policy priorities. For senior executives, this AI wave is not simply about technology adoption but about securing strategic control over data, chips, power, and talent for the next digital decade.

United States: Outpacing the Field with Capital and Infrastructure

Unprecedented Investment—Now and Planned:

The US continues to set the pace in AI, with approximately $90 billion in new AI, data center, and power projects already underway in 2024–25.

In a historic shift, the Stargate initiative—a joint venture between SoftBank, Oracle, OpenAI, and partners—will inject up to $500 billion in additional US-based AI infrastructure over the next four years (Bloomberg, Reuters). This project will create multiple new “megasites,” fundamentally reshaping both compute and energy landscapes.

These investments are on top of major expansions by Google ($25B), Meta, Microsoft, CoreWeave, and others, with Pennsylvania and Texas emerging as major AI infrastructure hubs.

Data Center and Power Build-Out:

  • US data center capacity is expanding at an unprecedented rate.
  • By 2035, data centers are projected to consume 8.6% of total US electricity, up from around 4% today (Bloomberg).
  • This growth is catalyzing upgrades in grid infrastructure and a pivot towards both renewables and natural gas.

Chips—The Nvidia Factor:

  • Nvidia remains the linchpin of global AI compute, with US tech giants in pole position for supply.
  • Ongoing export restrictions to China exacerbate global scarcity, turning Nvidia allocation into a critical competitive edge.

Talent—A Global Magnet:

  • The US is the global epicenter of the AI talent war, with inflated offers, multi-million-dollar retention bonuses, and major headhunting from around the world—especially the UK, Australia, Canada, and EU.
  • Smaller US firms and startups are squeezed for expertise and restricted access to premium Nvidia GPU clusters.

China: Massive Expansion Under State Direction

State-Backed Investment and Strategy:

  • China invested $98 billion in AI in 2025 alone (Reuters), with a further national five-year plan worth $138B targeting foundational research, infrastructure, and industry adoption.
  • By 2030, the projected size of China’s AI-adjacent markets could reach $1.4 trillion, with major new state-backed initiatives coming online each year.

National and Overseas Infrastructure:

  • The “East Data, West Computing” program networks over 250 data centers across the country.
  • New megaprojects in regions like Xinjiang and Inner Mongolia are leveraging renewable power to meet AI demands, while Chinese companies (e.g., Haoyang) are building large-scale AI centers overseas (Thailand, Malaysia) to secure global computing capacity.

Chip Shortage—A Persistent Barrier:

  • The ongoing US ban on high-end Nvidia chips is pushing China to source tens of thousands of banned GPUs for their desert-based megacenters (Bloomberg), while accelerating efforts to develop domestic alternatives. However, it’s rumoured that H20 chips from Nvidia will resume export to China soon, and Nvidia are developing a specific chip – the RTX Pro – for use in China’s smart factory and innovative robotics.

Talent—Scaling at Scale:

  • China is investing in mass upskilling through university-industry-government partnerships, but faces challenges retaining world-leading talent, with top scientists occasionally being drawn to opportunities abroad.

United Kingdom: Ambitious Policy and Regional Focus

Public and Private Commitment:

  • The UK government is targeting £14 billion (~$17 billion) in new AI investment by 2030, largely public sector-led but structured to encourage private match-funding.
  • Strategic “AI Growth Zones” are designed to increase UK compute capacity 20-fold, focusing on faster data center development and guaranteed energy supply.

Talent, Policy, and Research:

  • UK universities remain global leaders in AI research but are challenged by a notable “brain drain” of top experts to the US and (to a lesser extent) China.
  • The UK’s key differentiator is regulatory innovation and leadership in AI safety and ethics, promoting responsible development even as it strives to overcome scale constraints.

Australia: Fast Follower Status

Surging but Modest in Scale:

  • Total AI spending is expected to reach $3.6 billion in 2025, with leading sectors including finance, government, and services.
  • Amazon’s AU$20 billion (US$13B) investment in local data centers by 2029 is the anchor of Australia’s next-gen AI infrastructure, representing by far the largest digital investment in national history.

Talent and Capability Challenges:

  • Australia faces a skills gap of over 160,000 AI specialists by 2030, with local companies actively competing with global recruiters—especially US tech giants.
  • Partnerships, training initiatives, and public-private funding are ramping up, but substantive innovation at scale remains several years behind the US and China.

Comparing Regional Strategies and Future Investment

CountryPlanned AI Investment (2025 and Beyond)Key Focus
USA$500B+ (Stargate, 2025–29, plus $90B+ current)Largest global AI project, hyperscale data centers, energy buildout, and talent import.
China$98B (2025), $138B (2025–30), $1.4T target by 2030Largest state-driven AI expansion, domestic and overseas data centers, energy mega-projects, local chip and skills autonomy.
UK£14B (~$17B) by 2030Regional compute expansion, AI regulation, safety policy, and research; focused on homegrown growth and retention.
AustraliaAU$20B ($13B) by 2029 (Amazon) + public supportRapid cloud/AI infrastructure buildout, skills development, local ecosystem support; infrastructure led by global cloud giants.

Executive Takeaways

  • The US and China are locked in a “build-at-any-cost” race—deploying capital and resources on a scale that vastly outpaces other regions and reshapes national strategies on energy, data, chips, and workforce.
  • The UK and Australia are scaling up through targeted incentives, talent initiatives, and regulatory leadership, but must overcome both absolute funding gaps and challenges in talent retention.
  • SoftBank-Oracle-OpenAI Stargate ($500B) and China’s multi-year, trillion-dollar AI push signal an era where single projects or programs can eclipse whole nations’ historic tech spend.
  • Winning in AI for companies worldwide now means securing compute resources, reliable power, critical chip supply, and top-tier talent—each increasingly scarce and expensive.

Watch out for how these major commitments materialize, as actualized investment will determine which regions become the foundational hubs of global AI in the decade ahead.

Sources: This summary integrates data and reporting from Bloomberg, Reuters, national government releases, and recent executive announcements from the companies referenced. All forecasts are subject to change as projects are funded and executed.

SaaS and AI: The Future of Cloud Computing 2025

I found this draft I wrote this at the end of 2024 and had planned to publish, however time and events conspired against me. I think this is still valid and somewhat insightful – just bear in mind it was six months ago…

Thank you 2024, that’s a wrap! Over the year we’ve seen the switch from AI experimentation to development of some real use cases. A lot of development to incorporate AI into products. No doubt there will be more of this next year. For now the highlights of 2024 show us an evolution of the cloud computing model and the big players’ AI strategy execution.

The Cloud’s Favorite Child (SaaS)

Software-as-a-Service (SaaS) dominates the cloud conversation, with its market projected to reach $250 billion by 2025. Businesses are flocking to SaaS solutions like it’s a free buffet—who can resist the promise of streamlined operations and improved productivity? However, this rapid adoption isn’t without its quirks.

Challenges Ahead:

  • Integration Issues: With so many SaaS tools in play, companies will find themselves in a digital juggling act. Finding a way for these tools to communicate effectively will be key—after all, nobody wants their marketing software arguing with their finance app.
  • Security Concerns: As businesses embrace more SaaS solutions, the risk of cyber threats increases. It’s like inviting a raccoon into your pantry; it might seem cute until you realize it’s rummaging through your snacks.
  • Complex Pricing Models: The days of straightforward pricing are fading. Expect to see more usage-based models that could make budgeting as tricky as solving a Rubik’s cube blindfolded.

AI: The New Cloud Powerhouse

While SaaS is busy taking center stage, the big players—Microsoft, Amazon, and Google—are building foundational AI services that promise to revolutionize cloud computing. By 2025, AI will no longer be just an add-on; it will be the brain behind many cloud operations.

Key Trends to Watch:

  • AI-Powered Solutions: Expect AI to optimize everything from resource allocation to threat detection. It’s like having a personal assistant who not only organizes your calendar but also predicts when you’ll need an extra cup of coffee.
  • Edge Computing Integration: As IoT devices proliferate, edge computing will become essential for reducing latency and enhancing performance. This means data processing happens closer to where it’s generated—ideal for real-time applications like autonomous vehicles.
  • Multi-Cloud Strategies: Companies will increasingly adopt multi-cloud environments to avoid vendor lock-in and enhance flexibility. It’s akin to dating multiple partners until you find “the one”—except here, you can have your cake and eat it too.

The Road Ahead: What 2025 Holds

  1. AI Everywhere: Prepare for AI to be embedded in every aspect of cloud services, transforming how businesses operate and making processes smoother than a well-rehearsed punchline.
  2. Hybrid Cloud Solutions: These will gain traction as organizations seek to blend public and private clouds for enhanced security and flexibility. Think of it as having the best of both worlds—like enjoying pizza while on a diet (in moderation, of course).
  3. Focus on Security: With increasing threats, businesses will prioritize robust security measures in their cloud strategies. After all, nobody wants their sensitive data exposed like a poorly timed joke at a comedy show.
  4. Sustainability Initiatives: As environmental concerns grow, expect cloud providers to ramp up efforts toward greener solutions. It’s time for tech giants to show they can be eco-friendly without sacrificing performance.

As we step into 2025, the cloud landscape promises to be dynamic and full of opportunities. Embracing these changes will be crucial for businesses looking to thrive in this ever-evolving digital world.

The Impact of AI on Business Success

Artificial Intelligence (AI) has officially transitioned from the realm of science fiction into the everyday operations of enterprises around the globe. With a remarkable 72% of organizations reporting AI adoption across various functions, it’s clear that AI is not just a passing trend—it’s becoming an essential part of the business landscape.

A Surge in Adoption

Since the introduction of generative AI technologies, organizations have rapidly embraced these tools. A recent McKinsey survey indicates that 65% of companies are now using generative AI regularly. This shift highlights a growing recognition that AI can enhance productivity and drive significant business value—think of it as adding a turbocharger to your existing engine.

Why Should Directors Pay Attention?

For directors and senior management, investing in AI isn’t merely about keeping pace with competitors; it’s about gaining a strategic advantage. Companies leveraging AI are experiencing measurable benefits, including cost reductions and revenue increases in the areas where these technologies are deployed. Imagine being able to predict market trends faster than your competitors can say “data-driven decisions!”

Investment Opportunities for 2024

As we look ahead, here are some key areas where investments can yield substantial returns:

  • Generative AI Tools: These tools can automate content creation and customer interactions. Investing in these technologies allows your marketing team to focus on strategy while AI manages routine tasks, leading to greater efficiency.
  • Predictive Analytics: By utilizing AI to analyze data patterns, companies can anticipate customer needs and market trends. It’s like having an insightful advisor who can help you navigate complex decisions.
  • AI-Enhanced Customer Service: Chatbots and virtual assistants are now vital for improving customer satisfaction across industries. They provide timely responses, ensuring customers feel valued without overwhelming your human staff.
  • Smart Automation: From supply chain management to HR processes, automating repetitive tasks frees up human talent for more strategic roles. This shift not only enhances productivity but also boosts employee morale by allowing them to engage in more meaningful work.

AI is not Perfect!

While integrating AI may seem daunting, it often comes with its own light-hearted moments. For instance, employees might find themselves amused when their AI assistant occasionally misunderstands a request—turning “schedule a meeting” into “create a new playlist.” These quirks remind us that while technology is powerful, it still has room for growth. Additionally, concerns about job displacement due to AI are often balanced by the fact that many studies predict the creation of approximately 97 million new jobs by 2025. Rather than fearing robots taking over, we need to encourage a view of them as partners in enhancing workplace efficiency.

Embrace the Future

Closing out, the current state of artificial intelligence presents a significant opportunity for enterprises willing to embrace this technology. By investing in AI-driven solutions, directors and senior management can enhance productivity and push their competitive edge. Let’s look forward to a future where humans and machines collaborate effectively—working together toward success with a partnership focused on innovation and productivity.

Harnessing the Power of Teamwork: The Secret Ingredient to Successful Sailing

Sailing is a thrilling and adventurous pastime that has captivated enthusiasts for centuries. For many sailors, the allure of being out on the open water, feeling the wind in their hair and the sun on their faces, is an unmatched experience. But there’s more to sailing than just enjoyment – there’s a valuable lesson in teamwork. This past Saturday, I had the pleasure of racing as part of a crew on a sailboat, and it reaffirmed my belief in the power of teamwork not just on the water, but everywhere.

The Essence of Teamwork in Sailing

Sailing a boat, especially during a race, is a complex and demanding task that requires the seamless coordination of multiple roles. From trimming the sails and adjusting the course to monitoring the weather conditions and making tactical decisions, every crew member plays a crucial role in the success of the journey.

When you’re part of a sailing crew, you quickly learn that no individual can single-handedly manage the boat. It takes a collective effort, with each person contributing their skills and knowledge, to navigate the vessel efficiently and safely. The importance of teamwork in sailing can be distilled into three key aspects: communication, trust, and collaboration.

  1. Communication
    Clear and effective communication is the backbone of any successful sailing team. Crew members must constantly relay information to one another, from changes in wind direction to potential obstacles ahead. Timely and accurate communication is crucial in making quick decisions, especially when racing against other boats or dealing with unpredictable weather conditions. Good communication also helps to foster camaraderie, enabling the crew to work together more effectively.
  2. Trust
    Sailing a boat at high speeds and in challenging conditions requires a tremendous amount of trust in your fellow crew members. You need to have confidence that each person will fulfill their role competently and that they will have your back when you need support. This trust is earned over time, through practice and experience, and it’s essential to the crew’s overall success.
  3. Collaboration
    When a sailboat crew works together in harmony, the result is a finely-tuned machine. Each crew member must be aware of the others’ responsibilities and be prepared to step in and assist when necessary. This collaborative mindset is the key to overcoming challenges, adapting to changing conditions, and ultimately, winning races.

Being part of a sailboat crew has taught me the importance of teamwork in a way that few other experiences can. Out on the water, the stakes are high, and every decision counts. By fostering a strong team dynamic through communication, trust, and collaboration, a sailing crew can overcome obstacles, adapt to changing circumstances, and reach their destination successfully.

Whether you’re a seasoned worker or new to a workforce, never underestimate the power of teamwork. So, the next time you’re part of a team, remember to support one another, communicate openly, and work together – and you’ll find that success is smooth sailing.

Recent Developments in Cloud Computing: A Glimpse into the Future

Welcome back to MyTechStuff.site! In today’s post, we’ll explore the latest developments in cloud computing that are shaping the future of this fast-moving ICT industry. While cost optimization and security remain important considerations, we’ll focus on the exciting innovations and trends that are shaping up. Let’s dive in!

  1. Serverless Computing

The rise of serverless computing is revolutionizing the way businesses build and deploy applications. By eliminating the need to manage server infrastructure, serverless computing enables developers to focus on writing code and delivering value to their customers. This innovative approach allows for faster development cycles, better resource utilization, and automatic scaling based on demand.

  1. Multi-Cloud Strategies

As organizations seek to optimize their cloud investments and minimize vendor lock-in, multi-cloud strategies are becoming increasingly popular. By utilizing multiple cloud providers, businesses can leverage the unique strengths and capabilities of each platform, creating a more flexible and resilient cloud environment. This approach also allows organizations to distribute their workloads across multiple providers, ensuring data redundancy and reducing the risk of downtime.

  1. AI and Machine Learning Integration

The integration of artificial intelligence (AI) and machine learning (ML) into cloud computing platforms is enabling businesses to unlock new insights and automate complex processes. These technologies can help organizations analyze large datasets, identify patterns, and make data-driven decisions. Additionally, AI and ML can optimize cloud resource usage, helping you to adjust allocations based on demand or actual utilization, and reducing overall costs.

  1. Edge Computing

Edge computing is gaining traction as a complementary technology to traditional cloud computing. By processing data closer to the source, edge computing reduces latency and bandwidth requirements, improving the performance of data-intensive applications. This development is particularly important for Internet of Things (IoT) devices and real-time analytics, where low latency is crucial for optimal performance.

  1. Enhanced Security and Cost Optimization

Although not the primary focus of this post, it’s worth mentioning that security and cost optimization continue to be essential aspects of cloud computing. As the industry evolves, providers are constantly developing new features and tools to help businesses protect their sensitive data and optimize their cloud investments.

Conclusion

The cloud computing landscape is constantly evolving, with new developments and innovations shaping the future of the industry. From serverless computing and multi-cloud strategies to AI integration and edge computing, these recent advancements are transforming the way businesses operate and opening up new possibilities. As cloud computing continues to mature, it’s crucial for organizations to stay informed about the latest trends and adapt their strategies accordingly.

Remember to keep an eye on security and cost optimization, as these aspects will always be relevant in the world of cloud computing. Stay tuned for future posts on MyTechStuff.site, where we’ll dive deeper into these exciting developments and explore their implications for businesses.

Foundational DevOps

Benefits of Infrastructure-as-Code and Cloud Economics

As I see customers adopt Amazon Web Services, one of the first benefits they quickly realise is the ability to create and bootstrap environments at a time that suits them. This is a great benefit that helps to: (1) manage costs; and, (2)  enable experimentation of new ideas. It appeals from both a financial perspective and an engineering perspective. With this foundational capability in hand, an organisation can build on it to gain further benefits. For example, accelerating product development to gain a competitive advantage.

Environments in Traditional Data Centres

In a traditional data centre we would typically see a dev | test | prod | dr type approach to defining non-production (development and test) and production (prod and disaster recovery) environments. The infrastructure for these environments would be purchased at a high cost. Then it would often be written down, for example over a typical 3-5 year hardware refresh cycle. Guesses would be made to estimate capacity in advance of equipment purchase, and proof-of-concept work would typically occur just-in-time of purchase. Proof-of-concept in a hardware refresh cycle might trial and prove new application architectures at that time, perhaps not to be revisited until the next refresh.

Environments in AWS Cloud

Thank goodness we’re no longer confined to traditional data centres! With Amazon Web Services, you can create infrastructure and services without paying any upfront purchase costs. You pay for what you use, when you use it. What’s more (and even better), when you are finished you can destroy the infrastructure and services you provisioned and no further costs are incurred. (Note of course I’m not suggesting you destroy your production environments here, but highlighting the lifecycle capability of provisioning environments in cloud).

TRG Talk - Cloud - The Economics of Cloud Computing

Run a proof-of-concept whenever you want! Trial adoption of database-as-a-service like Amazon Relational Database Service (RDS) to reduce your database administration costs and improve service availablity! Introduce high-availability and self-healing compute infrastructure, with Amazon Elastic Load Balancing across Availability Zones and EC2 Auto Scaling!

Why Does It Matter?

Cloud providers such as Amazon Web Services have heralded changes that are nothing short of revolutionary. These changes contribute to the widely acknowledged current technological revolution – the Fourth Industrial Revolution. Globally we have seen the concept of cloud economics introduced to organisations and rapidly adopted. There’s now a more level playing field between smaller organisations and larger ones, which is accelerating innovation, disruptive ideas and products.

Underlying digital agility, innovation and productivity is IaC. Infrastructure-as-Code. IaC is a foundational capability of agile digital organisations. Using IaC you write the programming code to create your infrastructure and services. Once the code is written, the process is effectively automated.

Amazon Web Services provides CloudFormation and the Cloud Development Kit (CDK) for IaC.

Why use a human to do dumb, repetitive tasks? Automate them and boost your operational efficiency. Once you have your infrastructure code in hand, build a DevOps pipeline to manage the process of provisioning.

Foundational DevOps relies on IaC.

Imagine Who’s Listening

Humour me and imagine, if you will, some years in the not too distant future: you are part of a group putting your attention to understanding something. Somewhat like you might be doing now.

Boy on tin can phone listening to curious good news

Not everyone in your group is human.

We are not talking about alien life here. We’re not talking about animals either. No. Imagine for instance this particular fellow group member is an “intelligent agent”.

There are various definitions of an intelligent agent. For this case, let’s say that an intelligent agent is a device that perceives its environment and takes actions to maximise its chance of success.

An intelligent agent is a type of thing we more generally call artifical intelligence, or AI.

AI has arrived. What is AI?

AI and intelligent agents are changing the way we live and work.

Let’s think about this concept of intelligence. What is intelligence? Well, the Oxford dictionary defines it as “the ability to acquire and apply knowledge and skills”. Other definitions include: self-awareness, emotional knowledge, and capacity for logic. For our purpose of discussion, I’m going to define intelligence here as “the ability to achieve complex goals“. This definition broadly covers all of those ideas. There are many goals and the ability to achieve them provides a basis for intelligence.

Intelligence comes on a scale. The degree of ability to achieve a goal helps us understand where on the spectrum of intelligence something is. A complex goal might be: speaking. Let’s consider. Can a baby speak? No. Can an adult speak? Yes, generally. What about a child? All are somehwere different on the spectrum of intelligence.

Intelligence is narrow or broad. IBM’s Deep Blue chess computer beat the chess grandmaster Gary Kasparov in 1997. It could beat a chess grandmaster at chess but in noughts and crosses (a.k.a. tic-tac-toe), it couldn’t beat a 4-year old. It was built with a narrow focus of ability.

The more recent Google Deep Mind DQN AI system can play dozens of vintage Atari games at human level or better. This system is built to be able to apply itself to new goals. We might consier it has a broader capacity for intelligence.

Humans however, we as a species are so far unique in all the world with reagrd to intelligence. We are able to master a huge amount of skills. We can learn languages, sports, vocations and so much more. There’s nothing on the planet to rival us at this point in time.

General AI is coming

We are seeing accelerated breakthroughs and uses of AI in broader areas of our lives. Research and development in artificial intelligence has an endgame focus of general AI at a human level. Narrow AI will eventually evolve to become general AI. Whilst we do not know when this will happen (and some do question whether it will ever happen), there is no doubt of the advances and applications of AI.

Deep Mind has been able to learn many different games by using a positive reinforcement deep learning technique called reinforcement learning. This is a general unsupervised learning technique that computers use to teach themselves how to achieve narrow goals.

Using this technique, Google Deep Mind has been able to master and outplay human testers on 29 different vintage Atari games. Never sleeping, never resting and with no need to eat, computers can spend almost endless time learning how to achieve their goals in virtual reality, and then apply that knowledge when ready.

AlphaGo demonstrated strategic creatvity when it beat Lee Sedol, considered the top player of Go in the world in 2016. It was expected to take another decade before AI beat a human Go champion. AlphaGo went on to beat all 20 top players the year after it beat Lee Sedol.

AphaGo and Lee Sedol

To put this feat in context… there are many more possible Go positions than there are atoms in the entire universe. Therefore players rely heavily on intuition alongside conscious analysis. AlphaGo shocked the world by defying ancient wisdom by playing on the 5th line early in its 2nd game against Lee Sedol, and it went on to win the game. This was a demonstration of intuitive / creative play from a machine-based atrtifical intelligence.

AI is being widely used throughout our lives today

Natural language translation was not really considered possible when I studied AI back in the 1990s. As a student of computer science at university I recall discussing computation ability and we considered it as unable to cope with the ambiguities of natural langauge back then. Now we see natural language translation all around us. We are now seeing translation happen in real-time as well. What other examples do we see of AI all around us?

AI is being used in finance. Most stock market buy-sell decisions are now made automatically by computers. Algorithmic trading is the AI behind this. Algorithmic trading is used to help profitable trading. It allows resources to be efficiently allocated across the world at the speed of light.

In healthcare we are seeing changes in multiple areas, like diagnosis and surgery:

  • In 2015 a Dutch study showed that computer diagnosis of prostrate cancer using MRI was as good as human radiologists
  • In 2016 a Stanford study showed AI could diagnose lung cancer using microscope images better than human pathologists could
  • Machine learning is now used in medical research institutes such as Walter Eliza Hall Institute, for instance in predictive modelling for best outcomes based on analysis of genes, diseases and treatment responses
  • 2 million robotic surgeries have been performed in the US smoothly between 2000 and 2013 according to a recent report in 2017

We are in the midst of the 4th industrial revolution. AI is one of the catalysts in this step change for humaity.

AI is now a permanent part of our lives, changing the way we live and work

Let’s review what we’ve covered.

Artificial Intelliegence has arrived. What is AI? It is a narrow, limited ability to achieve a goal, at this point time (2021). The engame is a broader, more capable ability to achieve complex goals.

General AI is coming. Accelerated breakthroughs have shown advancements in intuitive, creative, strategic mastery of games. Virtual reality is now commonly used for reinforcement learning in training AIs.

AI is widely used today for the positive benefit of human society. In finance and healthcare we are seeing improvements that benefit humans.

Imagine who’s listening

Imagine some time in the not too distant future, you’re part of a group putting your attention to understanding something. Not everyone in your group is human…

Amazon Virtual Private Cloud

Amazon Virtual Private Cloud (VPC) is an abstract network service that allows you to create a virtual network of your own. Back when first introduced in 2009, it was a revolutionary concept that enabled the creation of a network of your very own – without you needing to own any IT hardware.

IoT smart cities

At present time of writing a VPC enables you to create a network address space using any IPv4 address range, including RFC 1918 or publicly routable IP ranges. The network can be between 16 and 65,536 IPv4 addresses in size. IPv6 is also supported.

The architecture of AWS Global Infrastructure means that your VPC spans multiple Availability Zones. It spans all Availability Zones in the AWS Region. Unlike many technology infrastructure providers, every AWS Region has 3 or more Availability Zones (AZ). AZs are geographically separated locations within an AWS region, connected by redundant fast fibre-optic data links.

You can learn more about the AWS Global Network here: AWS re:Invent 2016: Amazon Global Network Overview with James Hamilton

Within your VPC, you define subnets in an Availability Zone. This means whilst your VPC spans all AZs, your subnets will not.

To manage and secure network traffic flow you use route tables. A VPC is created with a main route table. Each subnet you create must be associated with a custom route table or the main route table. The route table defines routing for your subnet, indicating how network data should flow.

To further secure your subnets, Network Access Control Lists (NACLs) can be defined. A NACL can be used to explicitly Allow or Deny network data to cross the boundary into or out of your subnet. Each subnet must be associated with a NACL – either the default NACL (provisioned when your VPC is first created) or a custom NACL.

One more security feature for capturing network traffic flows is VPC Flow Logs. This allows you to capture the traffic that flows to and from the network interfaces in your VPC or subnet.

There is much more to VPCs than this but these are the fundamentals. You can create an AWS account and create and destroy VPCs either through a management console or programmatically.

There is some further reading here exploring options to extend your data centres to include VPCs: AWS Whitepaper: Extend Your IT Infrastructure with Amazon Virtual Private Cloud

PrivateLink – It’s a Kind of Magic

AWS PrivateLink is an interesting way to create an endpoint by which you can provide services to other AWS accounts. You can do this without the need to run requests through the Internet and without peering or otherwise “connecting” VPCs. What is this particular type of AWS magic, I hear you say?

Says the Amazon web site, “AWS PrivateLink provides private connectivity between VPCs and services hosted on AWS or on-premises, securely on the Amazon network. By providing a private endpoint to access your services, AWS PrivateLink ensures your traffic is not exposed to the public internet. AWS PrivateLink makes it easy to connect services across different accounts and VPCs to significantly simplify your network architecture.”

So this is a particularly interesting AWS magic trick in that we can provide services to other consumer VPCs, through the Amazon backbone. We simply do two things to make his happen:

  1. Create an Endpoint Service, in our serivce provider VPC
  2. Create an Interface Endpoint (linked to our Endpoint Service), in our service consumer VPC

Where this gets really interesting is that it avoids all the unruliness of network address spaces and having to deal with Network Address Tranlsation (NAT). Routing works through a network interface to the endpoint service and you don’t have to worry about the network addresses. And if the endpoint service is unavailable in one Availability Zone, well that’s not a problem because your endpoint service will load balance across multiple Availability Zones.

Not to put too finer point on it, but to get the engineering and provisioning underlying all that without lifting a finger? That’s a kind of magic.