I've chanced upon a similar issue and I'd like to extend the problem described above hoping to find some enlightenment.
In the same website linked above, there is a checkbox option (or a form input) for "Also find historicised data" to obtain a full history of the dataset. Upon inspection of the html element and checking out the codes above, this leads to a POST
to https://www.bundesanzeiger.de/pub/en/nlp?0-1.-nlp\~filter\~form\~panel-form with a payload of form inputs.
payload = {
"fulltext": None,
"positionsinhaber": None,
"ermittent": None,
"isin": None,
"positionVon": None,
"positionBis": None,
"datumVon": None,
"datumBis": None,
"isHistorical": "true",
"nlp-search-button": "Search net short positions"
}
Below, I'm using a modified version of Andre's code to POST
with isHistorical=true
, followed by doing a GET
of the original download link does seems to only return the default result (i.e. the non-historised dataset). I'm not too sure if there is something i might be missing here and would appreciate someone to take a look at this. Thanks!
import requests
def net_short_positions():
url = "https://www.bundesanzeiger.de/pub/en/nlp?0--top~csv~form~panel-form-csv~resource~link"
headers = {
"User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/118.0",
"Referer": "https://www.bundesanzeiger.de/",
}
payload = {
"fulltext": None,
"positionsinhaber": None,
"ermittent": None,
"isin": None,
"positionVon": None,
"positionBis": None,
"datumVon": None,
"datumBis": None,
"isHistorical": "true",
"nlp-search-button": "Search net short positions"
}
with requests.session() as s:
s.headers.update(headers)
s.get("https://www.bundesanzeiger.de/pub/en/nlp?0")
s.post("https://www.bundesanzeiger.de/pub/en/nlp?0-1.-nlp~filter~form~panel-form", data=payload, headers=headers, allow_redirects=False)
return s.get(url).content