UnauthorizedException is back

[peternijssen]

The UnauthorizedException is back like in the previous issue. Unclear is yet why, but apparently not everyone is suffering from this case, as noted in https://community.home-assistant.io/t/lovelace-postnl/112433/240

I tried to debug, but having a hard time to track my requests on an android device.

The earlier mentioned user agents here don't solve the problem. So it's unclear if it's related to the header.

cc @IcyPalm

[IcyPalm]

Yeah... PostNL made it a LOT harder. I appears that they are using: Akamai Bot Manager

Based on the X-acf-sensor-data header

I'm not sure if/how we are able to bypass this.

[peternijssen]

Damn... That sounds like they are actively blocking us....

[peternijssen]

USPS had the same case: home-assistant/core#13688

[IcyPalm]

Damn... That sounds like they are actively blocking us....

It's a possibility, if any PostNL devs are reading this: Let's have a beer and talk this over 🥂

[peternijssen]

I wonder if we set the update time to once a day, if we are being "unblocked" after like a week or so. Since it took some time for some people to get blocked.

[peternijssen]

I got around it.

Make the following changes in the code:

Change:

AUTHENTICATE_URL = BASE_URL + "/mobile/token"

To:

AUTHENTICATE_URL = BASE_URL + "/web/token"

Change:

"client_id": "pwAndroidApp",

To:

"client_id": "pwWebApp",

You are now using their web login, which still gives you an access key which also works with the mobile end points.

Potentially all end points could be rebuild to the web interface even....

[IcyPalm]

We have to check whether that is in the design specs of Homeassistant though, since it's borderline webscraping.

But I like the idea, I was just fiddling with the web api myself

[peternijssen]

Well, it's still an API. So not really scraping an HTML page. On the other hand, PostNL didn't build a "bot protection" without reasons I suppose.

[D2R9]

Well I still have trouble to login. I changed the lines as you said, but still received an 403 http error :-(

[IcyPalm]

I am working on a fix, currently trying out new ways to interface with the multiple endpoints

[iMicknl]

Did anyone have contact with PostNL regarding the API implementation? Possibly we could start the discussion and also discuss items like refresh time with them.

[peternijssen]

Tried through their webcare team, but failed. Was thinking about reaching out to them through Linkedin, so let me see what I can do.