Since the first try to get the Theia IDE and the Ionos Model Hub talking a few days have passed. This is partly because other projects have come in between. Out of a private need, a WordPress plugin has been created. With it travel stations from the popular Polarsteps app can be transformed into WordPress post. Another, more pressing reason, however, is that the solution to the problem with the "bad request" from the last blog article of my AI series was not as easily solved, as hoped for.
Just compile the Theia-IDE
One reason I value the open source community so much is because of its members. Known in English as a contributor, they ensure that large and small projects stay alive. As they did in this case as well. The GitHub Issue I had opened in regards to my problem, received a first answer after merely two days. Unfortunately, it wasn’t easy as setting the "developerMessageSettings" entry to "system". However, the answer was very close to the final solution. More on that later.
The next step in the search for the cause was that I was pointed to a version of the Theia IDE in which advanced logging of communication with the OpenAI endpoints is enabled. Unfortunately, however, this was only a GitHub branch, i.e. an adapted version of the IDE, which cannot be easily installed. What followed was a journey across the nightmare that is Electron. An Electron app is essentially a website that runs in its own small browser (Chromium), but appears like a normal program on the computer. Developers use web technologies such as HTML, CSS and JavaScript. Electron also ensures that the app has access to typical operating system functions, such as opening files or displaying notifications. This creates an application that feels like a real desktop app, even though it is technically based on web technology. This, by the way, is also the reason why the Theia-IDE can be used relatively easily as a "real" website. In the background, the project relies on NodeJS. Similar to other programming languages, NodeJS allows you to use functions provided by other packages for your own application. Since NodeJS was originally intended exclusively for use on servers, the language is often deeply integrated into the operating system. Unsurprisingly, the Theia-IDE could not be compiled directly. Instead, after some searching on the Internet, I found out that I still had to install the following packages on my Ubuntu laptop:
sudo apt install libxkbfile-dev libsecret-1-devAfter this, all dependencies installed successfully, but there were further problems. More specifically, compiling forced my student laptop to its knees. When I bought the laptop in 2011, NodeJS was just two years old and eight gigabytes more than enough. After a hint from the Github Issue and a "Vibe Coding" session *shutter* with Google’s Gemini, I was able to get to the goal through the following changes to the configuration files of the Theia IDE and overcome the resource hunger of NodeJS.
# examples/electron/package.json
"scripts": {
"bundle": "npm run rebuild && node --max-old-space-size=4096 ../../node_modules/.bin/theia build --mode development",
"start": "theia start --no-sandbox --plugins=local-dir:../../plugins --ovsx-router-config=../ovsx-router-config.json",# examples/electron/webpack.config.js
configs[0].cache = {
type: 'filesystem',
};
configs[0].devtool = 'eval-cheap-module-source-map';
configs[0].optimization = {
...configs[0].optimization,
minimize: false
};Debugging communication between the Theia-IDE and the IONOS OpenAI interface
The reason why all this was necessary is that this variant of Theia-IDE, once launched, documents communication between itself and an OpenAI interface. Specifically, it shows the user the HTTP request and also the response that the program sends or receives to IONOS. It became clear that after the request, that throws the error:
400 litellm.BadRequestError: OpenAIException - Error code: 400 - {'object': 'error', 'message': '', 'type': 'BadRequestError', 'param': None, 'code': 400} Received Model Group=meta-llama/CodeLlama-13b-Instruct-hf Available Model Group Fallbacks=Nonea second request – also sent to the IONOS-AI-Modelhub – returns successfully. So now we had to find out where the difference was between the two requests lies. At first, I assumed that the length of the statement sent to the AI model was too long and therefore led to problems. After some testing, however, this proved to be barking at the wrong tree. Instead, the solution – as already mentioned at the beginning – hid in the first note from the GitHub discussion: In the first request, the "role" parameter is set to "developer", in the second, successful request, however, to "user". Roles are used in the OpenAI API to prioritize the order of requests to the model. The successful "user" role is less relevant as the "developer" role. According to this forum post instructions in the "system" role – the original note from the GitHub issue – are now merged into the "developer" role. This is due to a change to the OpenAI-API in the end of 2024 and probably also the reason why the first answer did not immediately bring the hoped-for solution. Since both the "system" and "developer" roles resulted in the "bad request" error for requests to the IONOS AI model hub, I decided to instruct the Theia IDE in the settings, to use the "user" role instead of using the other two.
{
"ai-features.openAiCustom.customOpenAiModels": [
{
"model": "meta-llama/CodeLlama-13b-Instruct-hf",
"url": "https://openai.inference.de-txl.ionos.com/v1",
"id": "IONOS Code Llama 13b",
"apiKey": "eyJ0eXAiOi...Rest...Des...Tokens...Des...KI-Nutzers...aus...dem...DCD...",
"developerMessageSettings": "user"
}
]
}HEUREKA! The IDE’s "universal" agent responded to my "hello" with a number of things he could do for me. It can be that simple sometimes.
Searching for the cause – litellm in the IONOS-AI-Modelhub
A look at the original error code shows that unlike what I initially assumed, not the Theia-IDE, but the IONOS-AI-Modelhub use "litellm" in the background.
400 litellm.BadRequestError: OpenAIException - Error code: 400 - {'object': 'error', 'message': '', 'type': 'BadRequestError', 'param': None, 'code': 400} Received Model Group=meta-llama/CodeLlama-13b-Instruct-hf Available Model Group Fallbacks=NoneIf you browse around the litellm repository on GitHub, you will come across various issues which report the same error, albeit in a slightly different form. It can therefore be assumed that the litellm implementation that IONOS uses is currently plagued by similar bugs.
Conclusion: Sovereignty requires some dedication
Although I am pleased that I can now continue with my project "Sovereign AI-driven software development", it shows once again that some commitment is required if you want to break away from the big US providers. For me, unfortunately, this also means that we have October: from now on, I will pay for the requests to IONOS. In the data center designer, however, there seems to be no daily billing. This brings me back to one of my initial questions: How to avoid exploding costs? The answer: litellm. More on that in my next post.
