Author: mytechstuffsite

Chris Cormack is a seasoned professional with 30 years of IT experience, specializing in guiding leadership teams through cloud transformation and AI initiatives. As a trusted advisor to C-suite executives, Chris excels in translating complex strategies—such as cloud implementation and hybrid solutions—into measurable business outcomes. He identifies critical challenges and transforms them into opportunities for innovation and growth. With Chris's expertise, organizations can effectively harness the full potential of cloud and AI technologies, driving innovation and maintaining a competitive edge in an increasingly digital marketplace.

Publishing a .NET Core App

In my last blog, I mentioned a basic Web App that I had put together using Visual Studio 2015 tooling for .NET Core. Over the last couple of days, I’ve been looking at publishing the App, and the steps involved.

.NET Core is a new beast with a lot of potential…. the aim is it will run anywhere, on anything. So to keep my costs down, I’m going to trial it on my Amazon Linux VM. Note to self, SQL Server for Linux is about 1 year down the track – unfortunate, as the MVC scaffold uses SQL Server LocalDB – so I’ll have to figure out what database I can use.

But, first things first… how to publish my App to Linux?

First step – publish the App using dotnet publish. I found that Bower was not referenced in my Path environment variable. Bower was installed with Visual Studio 2015 Professional – in a sub-folder of my Visual Studio 2015 folder. I’m not sure if it was installed because I had installed tooling for .NET Core, or if it comes by default. Anyway, once that was sorted, dotnet publish worked fine and it created a portable for me to use.

I copied all of the files in the folder that dotnet publish created over to my Amazon Linux server. (I used WinSCP for this). Then I found I needed to install .NET Core on Amazon Linux. Installation was easy but when I tried to run the dotnet CLI, I received an error. Running dotnet –info from a bash shell I saw,

dotnet: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.18′ not found (required by dotnet)

After several hours of searching, I found this was due to the libstdc++ library version on my Amazon Linux distribution, which was libstdc++47. I had ensured my VM was up to date, so it seems that the libstc++ version was lagging (for whatever reason). After running the below, I was able to successfully run the dotnet CLI, with a valid response from dotnet –info.

sudo yum install libstdc++48

So, I had installed .NET core and fixed up the reference library it needed. After that, I needed a way to access my Web App. Kestrel is a web server built into .NET Core and can listen on any port you tell it to. However, it is better to use a web server as a proxy/reverse-proxy to relay request/response to Kestrel.

I already had a web server on my Amazon Linux VM. So I configured it with proxy and reverse-proxy mappings (to the Kestrel server in my .NET Core App). The calls to my web server forward on to the Kestrel server in the App, and vice-versa. I ran the App with dotnet run and checked access via the web server. All good 🙂

Almost there.

Finally, I installed supervisor to manage start/stop/restart of my App. And set up a script to ensure supervisor is re-started whenever my Amazon Linux machine restarts.

Note, this would have taken days if not for the early groundwork of several people who blog their efforts.

Now I have a Web App but no database. So next step is to get the database up and running.

Scrapy Spiders, Python Processing & Web APIs

Over the past couple of weeks, I’ve spent some time drafting a Web App for Touch Footy results. The App is built on .NET Core, and this gave me a great opportunity to review the new Visual Studio .NET Core tooling. But once I had my bare bones App, I needed some data to play with. Enter Scrapy and Python…

What I wanted was a data set that I could use in the Touch Footy App. I had a good data source and I figured my best bet was a web scraper. Scrapy made it easy for me to scrape together my test data set. It’s built using Python (hence you need some understanding of Python to use it). Python is an interpreted language. It’s great for list processing and it’s easy to read/write.

Scrapy is an open source framework for writing web crawlers, or spiders. It gives you control over how and when you execute the spiders you’ve written. And a great shell as a part of the framework to test/debug commands. After looking at other web scraping options, I decided on Scrapy as a neat way to get my data.

After a few hours coding, I had a crawler that collected the data I wanted, i.e. groups, teams, fixtures and results for my Web App. I wanted to store the data in JSON – for easy processing – and Scrapy made that easy too. It was simple then to write some Python code to process the JSON for groups, teams, fixtures and results.

All good so far, and fun to boot. My next step – how to get the data to the Touch Footy Web App? Well, Visual Studio 2015 tooling for .NET Core makes it easy to add a Web API to an MVC Web Application…. several hours later I had a working spider populating data into my App.

Spinning ever faster

The world’s not spinning any faster… but working in IT these days sees my thoughts spinning faster than ever. I’m a career IT guy and I have to stay on top of this stuff. Not just for my career, but it’s just who I am. It’s in my DNA.

I’ve done this tech stuff for a long time, and I’m pretty quick on the uptake. But it’s getting crazy out there. The rate of tech advance increases day by day, creating new or deeper specializations over time. It’s a time-consuming effort to maintain the general knowledge to manage IT well.

Mind you, that general knowledge is actually highly specialized tech knowledge. And very valuable it is too. So, in an effort to make it more accessible for myself, I’m gong to start writing down my thoughts. Maybe even organizing them.

And perhaps they will be useful not just for me. So here we go…