Ok I solved it. For anyone who had the same problem. I followed the tutorial using a devcontainer. i forgot to use the docker variable for my host network.
Changing the config to:
models:
- provider: self-hosted
model: openai/gpt-oss-20b
api_base: "http://host.docker.internal:1234/v1"
did the trick