Home Content Featured
Category:

Featured

ContentEnergyFeaturedLinksResources

Electricity Market Modelling Resources

by Jack Simpson
written by Jack Simpson

One of the amazing things about working in the energy industry is the sheer volume of reasonably well-structured datasets that are available. The real limitation when working with this data is the challenge of acquiring enough domain-expertise to know how to unlock its full potential.

Following on from my previous post, I’ve written this article to document some of the resources I’ve found helpful on my own journey. Most (aside from the physical textbooks) are freely available online, and I intend to continue to update and tweak the resources listed below in the future.

Energy Markets

Energy Modelling

Continue Reading
0 comments
0 FacebookTwitterPinterestEmail
ContentData ScienceFeaturedLinksMachine LearningResourcesStatistics

Simple explanations of data science and statistics concepts

by Jack Simpson
written by Jack Simpson

One of my favourite data science resources is the mini-episode series of the Data Skeptic podcast. These short episodes would feature the host explaining a data science concept to a non-expert in plain English.

I wanted to share a few of these with some colleagues from work and thought I’d catalogue them here.

Continue Reading
1 comment
0 FacebookTwitterPinterestEmail
ContentEnergyFeaturedVisualisations

Primary frequency response in action!

by Jack Simpson
written by Jack Simpson

On the 25th of May, an explosion and fire at Callide Power Station initiated a series of events that culminated in blackouts affecting hundreds of thousands of Queenslanders.

The figure below shows the coal power station units at Callide, Stanwell, and Gladstone tripping as this event unfolded between 1:45-2:10pm.

Power station units are in MW (left Y axis) while the system frequency units are in Hz (right Y axis).

However, when I examined the two Gladstone units that did not trip, I saw something rather interesting. As you can see in the figures below, in response to a sharp drop in system frequency, both units momentarily boosted their output.

Continue Reading
0 comments
1 FacebookTwitterPinterestEmail
PV
ContentEnergyFeaturedLinks

South Australian Gas Generator Interventions

by Jack Simpson
written by Jack Simpson

AEMO’s Quarterly Energy Dynamics (QED) report just came out today for Q1 2021. Of particular interest is the South Australian region which has been experiencing incredibly low prices thanks to the high levels of renewables in the market.

At the same time, these low prices have created system security challenges for the market operator. In fact over the past quarter, AEMO has had to intervene in the market a record 70% of the time to ensure that at least one gas powered generator remained running.

Continue Reading
0 comments
0 FacebookTwitterPinterestEmail
Startup Catalyst Collage
BloggingContentFeaturedTips & Tutorials

What I learned Visiting Silicon Valley

by Jack Simpson
written by Jack Simpson

Back in 2017, I had the amazing opportunity to be one of 20 young Australians selected by Startup Catalyst to receive a scholarship for their Future Founders Mission to Silicon Valley. On this trip I had the opportunity to visit dozens of tech companies including Google, Facebook, Twitter, etc and learn from senior people within these organisations. At the time I was constantly jotting down little notes, and I thought I’d take some of them and write them up as a post now.

Topics

Ideas

  • You’ll have more tries if you pick a market right. Really challenge yourself to imagine if there’s a lot of customers who will be willing to pay money. You can get things wrong but if you guess right about the market size then there will be acquisition options. Be realistic about growth and market size
  • Customer discovery is important: make sure you meet a need and the market is big enough to support your business.
  • Don’t get too emotionally attached to your idea/business and let it become your identity – being too attached can lead to you making bad decisions.
  • For every yes you get, you’ll hear no at least 100 times. When getting started, people will reject you many times. You need to learn not to take it personally, just keep trying.
  • Execution is important but so are exploration and discovery.
  • If you just have one machine learning algorithm you don’t have much, people with more money or data can kill you. It’s how you use machine learning to make your product better, stickier etc or if you can build up more data than anyone else and sell that data. It’s more how you can use machine learning to make your product better than machine learning in itself
Continue Reading
0 comments
0 FacebookTwitterPinterestEmail
ContentEnergyFeaturedVisualisations

Visualising the impact of Melbourne’s COVID-19 lockdown on Solar Panel Installations

by Jack Simpson
written by Jack Simpson

In July 2020, Melbourne entered its second COVID-19 lockdown – a lockdown which would not be lifted until late October. I thought it would be rather interesting to visualise the impact of this event on the number of PV system installations in each postcode of the city. These figures are based on the data provided by the Clean Energy Regulator which I merged with the spatial postcode data provided by the Australian Bureau of Statistics.

Pre-lockdown

Here’s how things looked in the months leading up to the lockdown – you can see how a few postcodes (in particular 3064 which appears to encompass Roxburgh Park) were having so many PV systems installed, I actually had to cap the maximum value of my heatmap at 300 so you could compare it to other postcodes in the figure.

Continue Reading
0 comments
0 FacebookTwitterPinterestEmail
Pandas Query
ContentData ScienceFeaturedProgrammingTips & Tutorials

How to improve the performance of Pandas with eval and query

by Jack Simpson
written by Jack Simpson

I should mention up-front that the techniques described in this post are really only worthwhile once you have a dataset in the millions of rows or above. Once your data hits this size, it is worth paying the initial optimisation overhead as it will save you memory and be faster overall.

Pandas’ eval and query is built on Python’s Numexpr library, and provides an optimised way to run a calculation or filter on a Pandas dataframe. For example, the code below shows you the traditional way of doing these things in Pandas:

start = '2020-02-10 08:20:00'
end = '2020-02-10 08:30:00'
duids = ['LYA4', 'BW02']

# traditional vectorized calculation
map_gen_df['DIST'] = np.sqrt(lya1_df['SEC_DIFF'].pow(2) + lya1_df['VALUE_DIFF'].pow(2))

# traditional filter
event_duid_df = map_gen_df[(map_gen_df['MMSNAME'].isin(duids))&(map_gen_df['TIMESTAMP_MIN']>=start)&(map_gen_df['TIMESTAMP_MIN']<=end)]
Continue Reading
0 comments
0 FacebookTwitterPinterestEmail
Conditional Code
ContentData ScienceFeaturedProgrammingTips & Tutorials

How to vectorize conditional calculations in Python

by Jack Simpson
written by Jack Simpson

Pandas and NumPy are fantastic libraries that enable you to take advantage of vectorization to write extremely efficient Python code. However, what happens when the calculation you wish to run changes based on the value in another column of your dataset?

For example, take a look at the dataset in the table below (along with the code to generate it):

GroupValue
A1
A1
B1
C1
import pandas as pd
import numpy as np

df = pd.DataFrame({
    'Group':['A','A','B','C'],
    'Value':[1,1,1,1]
})

Imagine I wish to create a third column (‘Result’) based on the following logic:

  • Multiply Value by 2 if Group == ‘A’
  • Multiply Value by 3 if Group == ‘B’
  • Multiply Value by 4 if Group == ‘C’
  • Fill with a missing value (nan) if none of the above is true
Continue Reading
0 comments
1 FacebookTwitterPinterestEmail
Power Lines
ContentEnergyExplanationsFeaturedTips & Tutorials

How to build a simple energy market dispatch engine

by Jack Simpson
written by Jack Simpson

This article will walk you through the construction of an extremely simple dispatch engine programmed in GMPL. While my preferred language is Python, GLPK lets us focus exclusively on the equations behind the dispatch engine, enabling us to understand the basics of how dispatch works in the energy market.

Background

So what is a dispatch engine? It is a program that allows us to figure out the optimial volume of power that different power stations should dispatch into the grid. As power is instantly produced and consumed, the total volume of power generated must equal demand at any time. Given that power stations sell different volumes of power at varying prices, the purpose of a dispatch engine is to identify how much each power station should generate to meet demand, while minimizing the total cost to supply that power.

Another important detail is that all power stations are ultimately paid the same price for the power they generate within a given interval – the most expensive power station called on to dispatch in order to satisfy demand effectively becomes the ‘price setter’.

Our dispatch engine is based on the technique of linear optimization – you can find out more background on the technique here if you’re interested.

Installing

Install GLPK (GNU Linear Programming Kit) – you can do this with the following command once you have Anaconda installed:

conda install -c conda-forge glpk

Parameters

We’re going to start by setting out all the parameters for our optimization:

  • We’re simulating a single region with a demand target of 150 MW
  • We’ll have 3 power stations – 2 coal, and 1 gas
    • Coal 1 sells power at $10/MWh and has a max output of 100 MW
    • Coal 2 sells power at $20/MWh and has a max output of 100 MW
    • Gas 1 sells power at $100/MWh and has a max output of 50 MW
Continue Reading
0 comments
0 FacebookTwitterPinterestEmail
Newer Posts
Older Posts