Chargepoint dashboard (aka fun with XML (and JSON))

[iainfm]

Hi, thought I'd venture out of the MG5 forum to talk about a little project I've got on the go.

The aim is to have a quick and easy way for me to see the status of the charge points in my area, so I can go and grab a charge when I need to.

I've got a quick 'n dirty powershell script that queries the DFT's feed, but it will probably end up as python or possibly javascript. It's in my github repository.

At the moment it just does a bit of XML processing based on what's returned from the DFT's API, eg:

ChargeDeviceRef ChargeDeviceName        ChargeDeviceStatus PublishStatusID
--------------- ----------------        ------------------ ---------------
SCO50690        Church Street Car Park  In service         1
SCO50691        Church Street Car Park  In service         1

ConnectorId ChargeMode ChargePointStatus
----------- ---------- -----------------
50690-1     3          In service
50690-2     3          In service
50691-1     3          In service
50691-2     3          In service

Interestingly the newest CP in my area isn't showing up yet so ChargeplaceScotland must have their own feed. I'll drop them a line to see if they have an API in the new year, because trying to scrape their web pages isn't the way to go.

I think I'll end up with something like my train departures board website, but will show the status of chargers in a town or postcode.

It's all dependent on my free time, but I'll pick away at it when I can.

Edit: Hmm, interestingly (and now one charger has gone into use) I can't immediately see where the 'in use' is being reported.

 

[Ozzie1989]

That's awesome, love doing a bit of programming in my spare time, just never get any

Look forward to see how this progresses

 

[ARealHuman]

Impressive! Also interested in how this develops

 

[Vipar]

Iainfm. Can you rewrite that in plain terms . I like to increase my knowledge but today I have found out that I know nothing
how does the charger info get into your spreadsheet?

 

[iainfm]

Iainfm. Can you rewrite that in plain terms . I like to increase my knowledge but today I have found out that I know nothing
how does the charger info get into your spreadsheet?

So... most [citation required] websites that present you with some sort of data link into a data source (ie the website is a way to display the data in a way that's useful to the consumer of it).

In this case the data source is some kind of database of charge point information. Where they are, what connectors they have, whether they're in use, how much they cost etc.

If you want to use that data for your own purpose/amusement you need to access it. Access is usually via an API, which is just a way that the developer has devised to let you ask for what data you want, and for it to return the data in a format that you can use.

In the DFT case, the API is simple - you just construct a URL defining what you want to know, for example:

https://chargepoints.dft.gov.uk/api/retrieve/registry/format/xml/post-town/Dumbarton/

If you create a web request either in a browser (by clicking the link) or programatically, you get returned a bunch of data in the format you specify - in this case XML. You can then find the bits of information you want to use and output them in some way (eg your own web page or export them to a spreadsheet - or just display them as the output from the script/program).

I've been trying to reverse-engineer the ChargePoint Scotland webpage today (their initial response to my request for API info wasn't what I was looking for), and managed to get a few hit-and-miss results. The complication with their API is that it requires authentication, and I don't have the info (yet) how they want the authentication to be made or what format to send requests in.

The other problem I'll have to face is the charge point statuses shown on CPS's website doesn't match what, say, ZapMap is showing - so I'll either need to find another single-point of information that covers all charge points accurately, or select the best data source based on location (which will be a bit of a ballache).

Anyway, I've re-sent my request to CPS, so hopefully they'll provide the documentation required for their API when they return.

 

[Vipar]

Thank you. Not thick mud anymore. More like muddy water I guess as time goes on I will understand the supplied information gradually and hopefully retain it. Thanks again for being patient with an old codger

 

[iainfm]

A little bit of progress this evening. I've got CPS's authentication working, so can now query the connection statuses of all the network's chargers.

At the moment I'm having to pull the whole lot as I don't know how to specify certain ones, and then filter them in code...which is very messy as I'm not well up on this kind of parsing malarky at all.

Anyway, I can now check on my favourite charge points by running a script instead of all that tedious web/app nonsense The output (currently) looks like this:

PS D:\Git\ChargePointDashboard> D:\Git\ChargePointDashboard\cps.ps1
60238 1 AVAILABLE
60238 2 OCCUPIED
50690 1 AVAILABLE
50690 2 AVAILABLE
50691 1 AVAILABLE
50691 2 AVAILABLE

Changes have been pushed to my git repository. Anyone playing along will need an API key for the CPS one to work. <whispers> It's in the page source for the main chargeplacescotland.org website, under 'apiAuthKey'. Which it possibly shouldn't be, but hey.

 

[iainfm]

Had a reply from CPS syaing they no longer provide API access/details, so I guess we'll do it the hard way.

 

[iainfm]

Man, I love Chrome's 'inspect' capabilities!

curl "https://account.chargeplacescotland.org/api/v2/poi/chargepoint/dynamic" ^
  -H "Connection: keep-alive" ^
  -H "api-auth:<redacted>" ^
  -H "Accept: application/json, text/javascript, */*; q=0.01" ^
  -H "sec-ch-ua-mobile: ?0" ^
  -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" ^
  -H "sec-ch-ua: ^\^" Not A;Brand^\^";v=^\^"99^\^", ^\^"Chromium^\^";v=^\^"96^\^", ^\^"Google Chrome^\^";v=^\^"96^\^"" ^
  -H "sec-ch-ua-platform: ^\^"Windows^\^"" ^
  -H "chargePointIDs: 50691" ^
  -H "Origin: https://chargeplacescotland.org" ^
  -H "Sec-Fetch-Site: same-site" ^
  -H "Sec-Fetch-Mode: cors" ^
  -H "Sec-Fetch-Dest: empty" ^
  -H "Referer: https://chargeplacescotland.org/" ^
  -H "Accept-Language: en-GB,en-US;q=0.9,en;q=0.8" ^
  --compressed &

Simply send it a list of comma-separated 'chargepointIDs' and out they pop.

PS D:\Git\ChargePointDashboard> D:\Git\ChargePointDashboard\cps.ps1
50691 1 AVAILABLE
50691 2 AVAILABLE

 

[iainfm]

Ok, for viewers in Scotland here is a proof of concept web app available at iainfm - Enter ChargePlace IDs.

Simply enter the ChargePlace Scotland charger IDs (comma separated) and hit 'get status', You'll get a list of ports on the requested chargers along with their current status.

It's rough and ready, but it'll satisfy my immediate requirement. It'd be nice to have the list sorted, and with the location of each charger but that's for another day. Maybe.

It's scaled as it is so that it displays nicely on a smart phone. Once you've got the screen showing your favourite chargers you can add it to your homescreen for easy reference.

Suggestions / feedback welcome, although I'm probably not going to do much more with it as a) time and b) it could break at any time if CPS change their API.

Until then, enjoy!

 

[admiralross2400]

This is really cool! I'm going to save it.
Is there any way to add the socket name at all? Depending how many you have it could get hard to remember all the locations

 

[iainfm]

Thanks!

Sockets themselves don't have names (just a number), but the charging units (the things that have one or more sockets) have an address that could be looked up (see, for example, Network News | ChargePlace Scotland - all the info presented here it technically get-able).

The slight problem is the API call that returns the 'in use' (or not) statuses doesn't give you the address, so you'd have to do two lookups and then match them on the ID. Which is definitely do-able, and something I'll have a go at.

This kind of data grabbing/processing isn't something I know very well, but with enough time and googling I can probably manage it!

Edit: In fact, it looks like the CPS website simply pulls down the static data for all the chargepoints, and then merges that with the availability information (which can be requested on a per-chargepoint basis). Which, I think, simplifies things a bit

Edit 2: Of course, it will be very dependent on the quality of the data being returned...

streetnumber :
street       : Falkirk
area         :
city         : James Street Car Park
postcode     : FK2 9PX
country      : GB

streetnumber : Eastside, Kirkintilloch
street       : Glasgow
area         : 
city         : Eastside
postcode     : G66 1QA
country      : GB

 

[iainfm]

As a quick enhancement I've made the charge point IDs into hyperlinks that take you to CPS's details page for that charger.

Note: for 'unavailable' chargers this may lead to an error 'Static CP data not found', but in-service ones seem to be ok.

To get the new version, simply refresh your bookmarked page or load a new set of IDs.

 

[iainfm]

A relatively minor update for today - sorting implemented and a slight change to the colour scheme.

Interestingly, it's showing the new rapid charger near me is now in service, so I'll need to check that out in due course

I think for the address lookup I'll pull down the static info daily, then reference it locally to reduce overheads.

Update pushed to the git repo.

 

[iainfm]

Today's update - address info for the CPs added as per @admiralross2400's suggestion. Layout alterered to suit (and look nicer).

The address info (once I automate it) will get pulled down daily (probably), so there may be a delay for any new points to appear. The usage info will be there; it'll just be the address that's missing.

As above, the address info for the CPS chargers is a bit shoddy in places, and I'm not responsible for its accuracy...or indeed the accuracy of any data presented. Use it at your own risk etc etc

Github repo updated. And yes, I know I don't comment my code enough! I'm from a time when every byte mattered

I think one further embellishment would be to add the charging cost, if applicable, seeing as a lot of them are no longer free. Until I do that you can get the info by tapping/clicking the charger ID to take you to the CPS details page for that unit.

 

[admiralross2400]

Today's update - address info for the CPs added as per @admiralross2400's suggestion. Layout alterered to suit (and look nicer).

The address info (once I automate it) will get pulled down daily (probably), so there may be a delay for any new points to appear. The usage info will be there; it'll just be the address that's missing.

As above, the address info for the CPS chargers is a bit shoddy in places, and I'm not responsible for its accuracy...or indeed the accuracy of any data presented. Use it at your own risk etc etc

Github repo updated. And yes, I know I don't comment my code enough! I'm from a time when every byte mattered

I think one further embellishment would be to add the charging cost, if applicable, seeing as a lot of them are no longer free. Until I do that you can get the info by tapping/clicking the charger ID to take you to the CPS details page for that unit.

Legend!

 

[iainfm]

Today's update: I've added the cost per kWh (if not free) to the info displayed. Note that this does not include any connection, 'waiting' or other fee (if applicable), so check the main CPS page for full details. You can always click/tap the charger ID to get straight to the official info.

Because of this, and general concerns around the quality of the source data, I've added a disclaimer at the bottom, which absolutely no solicitors have assisted with.

Please don't sue me!

Edit: Turns out CPS don't list the connection charge on the individual charger info either (that I can see). I did add it to a test version of the page, but the length of the info messed up my formatting, so I'll need to re-think the CSS if I'm going to use it.

 

[iainfm]

Hi,

Don't know if anyone is using this, but it has a new home on hosted webspace - complete with SSL certificate.

Please update your bookmarks to:

Main site: iainfm - Enter ChargePlace IDs

Favourites (for example): ChargePoint Dashboard

The old address will continue working for a little while (it redirects to the new one).

 

[iainfm]

I've made a few smallish changes to this -

A better (I think, but let me know) style, complete with icons for the connector types along with charging rate info.

Sharing options in case you want to, you know, share the info with anyone else.

I see ChargePlace Scotland have branched out into the Manchester area(!), so the chargers there (with their new-fangled alphanumeric identities) are supported now as well.

 

[jamier]

Hi Iain
This is great. Thanks
How easy would it be to get icons on a map or have a phone app or have push notifications when a charge point became available?
Kind regards
J