v0.9 Documentation
This project came out of the attempts to leverage AI to simplify doing statistical research into a large private database of clinical history.
The first efforts were frustrating in the extreme,but we learnt a lot about AI systems and the ways that they fail to live up to the hype surrounding them.
But at the same time, learning to work around those issues suggested a way to exploit AI technology for aspects of the research that it can be good at, and avoid the major consistency issues that AI has.
It turns out that database technologies form a reliable basis for repeatable, re-producible research analysis that AI does not directly do.
But AI can provide a massive level of skill set support for:
database structural design and revision to facilitate statistical analysis
proposing appropriate statistical methods for looking at your data
proposing presentation and visualisation methods
helping create programming objects to implement your choices
However, like many other people, we found major issues with relying on AI systems.. making the sarcastic version of the AI acronym (Artificial Idiot) all too accurate.
AI’s do not seem to deliver reproducible results .. they apply ‘creative’ revisions every time they process an interaction .. so you get a different answer to the same question. Because the context of that question changes even though the question itself is unchanged. At the very least, the context changes in a simple way – it is the second time you asked that question. And AI’s presume that the last answer was not acceptable – so they generate a new answer.
We have had one example (AnythingLLM using the using a qwen-3 model) simply invent data that should have been extracted from the database it was querying.. The customer number was the primary key of the table concerned... and query was extremely simple … just to list some fields from the table. The LLM just made up random data .. and kept doing it even when it was repeatedly informed of the problem. So all output from an LLM needs verification – they are unreliable.
What neither AI nor databases can do is know what is of interest for being researched … but they can enhance the capabilities of the person who has that interest.
The informed and interested researcher with domain specific expertise remains essential. This workbench is a tool to combine AI platforms, databases and software development to help the researcher.
Random note: the workbench is designed for normal desktop (or notebook) browser. No one can do any serious work on the small screen of a mobile phone.
We were looking for a workbench type solution for ourselves… and could not find one , so we built it.
It was created to meet the needs of a client who has limited IT skills, uses AI but only as a user not a developer, and is only moderately Database literate. They are well aware of statistical analysis, buy but from the perspective of asking statisticians to help them. But they are an absolute world class expert in their specialist domain – medical research.
They envisioned just chatting with an AI and it doing all the needed ‘grunt work’ to provide statistical analysis and visualisations.
Normally that would use a team of IT specialists, data scientists, and AI systems experts, but as a solo researcher, with an impressive historical database to research, that was a budget impossibility.
The workbench satisfies these needs by encapsulating in one tool, the most common parts of those skills under a web front end… and to gain the widest audience, is directed at the three most common operating systems – Windows, MacOS ad Linux. Our client uses Windows
We only found one tool that came close Auto Analystics but whilst it suits some use cases, and is offered as both source code and as a service, unless you install it on your own hardware, it can nodd meet the need for total data privacy that our client has (and many other people have)
We tried out installing it own hardware, but that requires extensive IT skills (which we have, but our client does not). Because the AI capability requires some serious hardware support, Auto Analytics is heavily focused on providing the AI aspects via the main AI providers … and that in turn requires that for the hosted platform they offer their software needs a comprehensive comprehensive cost recovery billing system. That is of course not a direct concern for anyone running their own site to ensure data privacy.. so the workbench leaves the billing for outsourced AI capacity with the AI Provider(s) that you decide to use. And there is no billing required if you use your own AI hardware. Auto Analytics has a very sophisticated multitasking design but the code as written can not run reliably in a Windows environment.
But the difficulties we had creating a local version of Auto Analytics also highlighted the need for a good Installer for our workbench.. One that deskills the software installation process. And there are planned to be three Installers – to cope with the differences between Windows, MacOS and Linux it is simpler to use different Installer software in each environment.
That allows us to base each installer on the ‘best of breed’ installer package tool in each system, and use the best choice package management in each system for follow on upgrades.
We see the AI assisted data science world being a parallel to the 1990’s when CPM and DOS brought computing out the computer centre and into general use by many businesses and small groups.
You could do almost any function from the command line, but it had a high learning curve.
Then Windows 3.0 and the early Apple systems introduced widespread use of the GUI interface approach. Primarily the advantage was you didn’t need to ‘know’ the commands, but could ‘select’ the action from the relevant choices shown to you. A much lower learning curve.
The GUI approach is really good for most cases, but can be overdone … many very low level structures need expertise – and just presenting them in a GUI does not remove the complexity that underlies things … so even an IT expert will still use command line tools for exotic problems, but use a GUI when only common changes or setups are needed.
Data analysis is working like the DOS world did – and covers100% of what can be done.
The workbench handles 80% of the data analysis that people would like to do, and does it via a web interface to reduce the skill set requirements . To focus on the desired outcome.