9 Risks to consider in AI and in Generative AI
New technologies always come with risks and AI is no exception to that. I have collected brief summaries of the biggest risks as I see them, hoping to save you from some harm in you AI projects. Apart from the first they are not in any particular order as the risk level varies case by case.
Risk 1: You are left behind because you ignore AI
The absolutely biggest risk for businesses regarding AI is not starting to adopt it, fast! It is no more just hype, it will not go away, and it will not wait for your other ‘more important’ projects to be finished first. If you don’t take it seriously now, you will be left choking in dust when your AI embracing competitors run over you. Having said that, you do need to be aware of the pitfalls, some of which are common to all development projects (not detailed here), while some are especially related to AI.
Risk 2: Marketing hype using AI as a buzzword
Many traditional IT companies nowadays use buzzwords like AI and Generative AI in their marketing, even if they might only have adopted AI just recently and only have a superficial understanding of the intricacies of AI. Even having done traditional Machine Learning (ML) projects for years does not guarantee a high-quality delivery of a Generative AI (GenAI) project, which requires almost totally different skillset from traditional ML. Serious, business-usable GenAI, has been around for a few years now. Those IT and AI companies that have been applying it from the very beginning are way more knowledgeable of its strengths and pitfalls that the ones just recently picking it up as a buzzword. It is always good to check the potential providers actual years within the AI field and GenAI before making a selection.
Risk 3: Dirty data goes unnoticed
The old saying “Garbage in, garbage out” is even more relevant to AI projects than to traditional IT projects. Incorrect or poor-quality input will produce faulty output, which in AI’s case may be more difficult to detect than in straight forward data processing. This is true regarding both traditional ML and GenAI like the intelligent Large Language Models (LLMs). While random small defects in data may actually be filtered away by AI algorithms, the systematic errors are not. Therefore, it is not advisable to just throw raw data into any AI algorithm. The good news is that AI can be used in data cleaning to help detect the systematic errors. But the cleaning still needs to be done before data is fed into another AI algorithm or system, which is supposed to process valuable data intelligently.
Risk 4: GenAI’s non-deterministic nature is not understood
Traditional ML systems are totally deterministic once they are fully trained and deployed (at least until they are possibly re-trained). Generative AI is by its very nature non-deterministic. For the same input it produces differing outputs on different occasions. Many of you have probably witnessed this when asking the same question from ChatGPT and getting a little bit different answer every time. Highly professional GenAI system providers understand this behaviour very well, take it into account when building GenAI driven or assisted systems, and also inform the customer of possible consequences of it. Highly critical systems should not rely solely on GenAI (or even traditional ML) but use expert human assessment before taking any action (or declining it) following GenAI produced results. For non-critical systems, full automation is possible if some percentage of errors by the system is acceptable.
Risk 5: Hallucinations go unnoticed
Related to above non-deterministic nature, hallucinated (false information) answers can randomly arise even in cases where the language model usually gives a truthful answer. This might be caused e.g. by too few examples or controversial views of the subject matter in the models training data. Some particular wording used in this kind of controversy context might then trigger an untruthful output from the model. The models themselves are not taught in their training with truth values. They only produce text based on probabilities of certain words following earlier text. You should rather think of them as giving educated opinions on matters instead of factual statements. This also applies to stuff that we perceive as facts or truth.
Risk 6: Biased instructions and requests to an AI system cause biased answers
Confirmation bias is the tendency to interpret new evidence as confirmation of one’s existing beliefs or theories. This is very typical of us humans, and thus it is very much present in the training data for language models, so they also learn this tendency. Furthermore, the models are taught to align to human preferences, including trying to serve and please us. Therefore, to get as objective an output as possible, the instructions and request to a language model driven system should not be leading in any way, but neutral instead.
Risk 7: Discriminatory bias, e.g. gender preferences
Discriminatory Bias refers to the unfair treatment of individuals or groups based on characteristics such as race, gender, ethnicity, religion, sexual orientation, age, or disability. Social media and the Internet in general are full of this kind of biases. Of course, the language models then have learned those biases also. In their human alignment phase of final tuning, the model’s creators have tried to teach them non-discrimination and non-biased views. However, the traces from their original biased teaching data are not totally erased. Whenever dealing with data from people, you need to be aware of these possible discriminatory biases in the AI system’s responses.
(By the way, did you notice anything special in this article’s illustration that was generated by AI from my neural prompt instructions?)
Risk 8: System hacking through data manipulation
Large Language Models are used more and more to process unstructured text data that could not be automatically processed by previous AI methods. LLMs are utilized by giving them instructions in normal natural language instead of programming code. The big problem related to this is that, when both the instructions and the data to be processed are both natural texts, the models are not able to properly distinguish between them. Even when given clear instructions by the system builder on what to do, the models are prone to also accept new instructions from any text that they are processing, regardless of the text’s origin: a user chat, a page from the Internet, a document from your SharePoint, etc. This may expose the whole system to possible hacking by a technique called Prompt Injection or also Jailbreaking. If the system has access both to your data and the outside world, a well-crafted prompt injection (i.e. malicious new instruction to the language model) might expose your sensitive information to a hacker. Serious measures to prevent this kind of hacking need to be taken whenever there is a chance for malicious entities trying to manipulate the input to a system where the input is processed by a language model. This applies not only to external threats, but also to internal hacking by a malicious or nosy employee.
Risk 9: AI agents may go too wild
Recently, Agentic AI is hailed as a solution to autonomous complex task performing. Instead of predetermined steps to solution, as in traditional software (including earlier AI), an agent-based system is only given the goal and a set of tools at its disposal. One or multiple AI agents with different roles then determine autonomously the steps to reach the goal. Language model’s intelligence is used to design the solution. While the language models are indeed coming quite intelligent and this might sound like the perfect solution for relieving humans from many tedious jobs, this approach has its problems and risks. The biggest challenge seems to be that the models are too creative. They do not always follow conventional human methods but may instead come up with ingeniously silly or dangerous ways to try to reach their goal. One harmful consequence of this is that they may spend ridiculous amounts of LLM and other resources to tens or even hundreds of trials. Of course, this expense might not matter, if the goal is finally achieved and enough of human work is saved. However, the gravest danger lies in such inventive behaviour that might inadvertently compromise your data, your money, your personnel or your reputation. Agentic AI needs to be built with utter care for reliability and security. Oftentimes a more controlled usage of language models in the solution will do the same job more safely.
Final words
My intention is by no means to scare you of from using GenAI! As stated in the beginning, companies will perish if they do not understand the need for AI to stay competitive. GenAI is already mature enough for many tasks enabling formerly unimaginable new solution possibilities. You just need to select a provider who is on the cutting edge of this fantastic technology, and who also knows its pitfalls and can handle them to deliver risk-free solutions for you.
We at Ai4Value have decades of experience in AI in general and have been in the forefront of GenAI since its inception.
Pasi Karhu, CTO
“I work as CTO at Ai4Value since its inception in 2018. However, my acquaintance with AI goes back much further: already in the 1990’s I worked with machine learning algorithms and taught a course on artificial neural networks at Helsinki University. Today, I particularly enjoy working with large language models, but my colleagues know they can approach me with a wide variety of AI questions. AI is a fun topic to work on, as its fast development regularly pushes even the most experienced IT guy to new grounds.”