First of all, let me describe the problem. I am a software developer, and I don’t use AI to write code just because of hallucinations. For creating pictures or writing texts they are not so critical, but for the task of writing code, they are overkill. I have identified two subproblems. Firstly, it is difficult to identify the errors made by AI. It is quite easy for a designer as he counts the fingers in the pictures and that’s it; the picture is accepted as the result.
However, I have to deal with the code and seek the mistakes in it. I hate to do it. Secondly, it is the irreproducibility of the result. For example, I tried to write unit tests: the first generated unit test is successful, but the second one is not, even for the same method. This makes working with it impossible.
Now, about the solution. Our hypothesis was as follows: what if we could assign the corresponding result data to a certain set of input information? And do that at least sometimes during the task-solving process. Would that reduce the number of hallucinations? Oh yes! After thorough research, we have come to the conclusion that the presence of a critical mass of such ‘rigidly fixed’ nodes almost completely removed hallucinations from the output. All that remained was to find the criteria for identifying these correspondences between input and output information and to modify the math apparatus to enable the use of this approach. We called this method the “preset landscape.”
Certainly, our approach has several limitations. First of all, the subject area should imply the existence of only one correct answer for a given set of output data. Fortunately, software development is exactly such an area. The second limitation is that at the stage of landscape formation, the participation of an expert in the subject field is required. In other words, it is impossible to apply the math apparatus to the areas where a human cannot articulate the rules. These limitations greatly narrow the scope of application of our approach. However, its usage in such areas as software development, law, healthcare, and engineering tasks is more than enough.
To demonstrate the capabilities of the math apparatus, we have developed a plugin for IntelliJ Idea (JetBrains). You can install it and make sure that there are no hallucinations. Here, you can find the instructions.
While we were working on the plugin, AI services came into vogue, which provides API and you can use them in your own solutions. Therefore, we are planning to make such a platform for software development tasks. We will most likely start with Java. If you have an insight into how this approach can be implemented for lawyers or healthcare professionals – feel free to share.
