Estimated Read Time: 40 minutes

HISTORY OF GENERATIVE AI

Gen AI is marked by several key developments and milestones.
In the 1980s, data scientist to move beyond the predefined rule and algorithms of traditional AI.
Later in the 1980s and 1990s came the introduction of models such as Hopfield Networks and Boltzmann machines => creating neural networks capable of generating new data.
Scaling up to large datasets was difficult and issues such as the vanishing gradient problem made it difficult to train deep networks.

In 2006, the Restricted Boltzmann Machine (RBM) solved the vanishing gradient problem, making it possible for deep neural network.
In 2014, the generative adversarial network (GAN) – to generate realistic data-> images.
Variational autoencoder (VAE) was introduced, offering a probabilistic approach for generating data.
Natural Language Processing (NLP), Generative pre-training transformers (GPT) and Bidirectional Encoder Representations from Transformers (BERT) revolutionised NLP
Generative AI is a vibrant field to evolve, with newer models like GPT-4, and DALL-E pushing the boundaries of what AI can generate.

WHAT is GENERATIVE AI ? Gen AI ?

Generative AI refers to artificial intelligence models designed to generate new content in the form of written text, audio, images, or videos.

Generative AI can be used to create a short story based on the style of a particular author, generate a realistic image of a person who doesn’t exist, compose a symphony in the style of a famous composer, or create a video clip from a simple textual description.
Gen AI can learn from data

OTHER TYPES OF AI

• Traditional AI

AI systems -> that can perform specific tasks by following predetermined rules or algorithms.
Rule-based systems that can’t learn from data or improve over time.

• Machine learning

ML enables a system to learn from data rather than through explicit programming.
Process where a computer program can adapt to and learn from new data independently, resulting in the discovery of trends and insights.
Gen AI makes use of machine learning techniques to learn from and create new data.

• Conversational AI

Enables machines to understand and respond to human language in a human-like manner.
While Gen AI and conversational AI may seem similar – particularly when generative AI is used to generate human-like text.
Conversational AI is used to create interactive systems that can engage in human-like dialogue.
Purpose of the Gen AI is different
Gen AI is broader, encompassing the creation of various data types, not just text.

• Artificial general intelligence (AGI)

Highly autonomous systems – currently hypothetical – can outperform humans at most economically valuable work.
AGI would be able to understand, learn, adapt, and implement knowledge across a wide range of tasks
While Gen AI can be a component of such systems, it’s not equivalent to AGI.
AGI denotes a broader level of autonomy and capability.

Gen AI focuses on creating new data instances.

Siri
ChatGPT
Grammarly
Quill Bot
Alexa
Gemini
Blackbox
DeepSeek

What is the capability of Gen AI ?

Designing virtual assistants
Generate human-like responses
Developing video games with dynamic and evolving content
Generating synthetic data for training other AI models, especially in scenarios where collecting real-world data might be challenging or impractical.
It can drive innovation, automate creative tasks, and provide personalised customer experiences.
Gen AI as a powerful new tool for creating content, solving complex problems, and transforming the way customers and workers interact with technology.
Generative AI is already having a profound impact on business applications.

HOW does GEN AI WORKS ?

Generative AI works on the principles of machine learning, a branch of artificial intelligence that enables machines to learn from data.
Generative AI — it not only learns from data but also creates new data instances that mimic the properties of the input data.

HOW GEN AI WORKS – Workflow

• Data collection

A large dataset containing examples of the type of content is collected.
Dataset of images for generating realistic pictures, or a dataset of text for generating coherent sentences.

• Model training

Gen AI model is constructed using neural networks.
The model is trained on the collected dataset to learn the underlying patterns and structures in the data.

HOW GEN AI WORKS

• Generation

Once the model is trained, it can generate new content by sampling from the latent space or through a generator network depending on the model used.
The generated content is a RESULT of what the model has learned from the training data.

• Refinement

Depending on the task and application, the generated content may undergo further refinement or post-processing to improve its quality or to meet specific requirements

hOW GEN AI WORKS

Cornerstone of Gen AI is Deep Learning

Deep Learning a type of machine learning that imitates the workings of the human brain in processing data and creating patterns for decision-making.
Use complex architectures known as artificial neural networks.
Neural networks comprise numerous interconnected layers that process and transfer information
Mimics a brain’s neural networks to learn from large amounts of data, enabling machines to solve complex problems.

TYPES OF GEN AI

• Transformer-based models

For text generation, transformer-based models such as GPT-3 and GPT-4
Entire context of the input text is considered, enabling them to generate highly coherent and contextually appropriate text.

• Generative adversarial networks (GANs)

Two parts, a generator and a discriminator.
The generator creates new data instances
The discriminator evaluates these instances for authenticity
Discriminator tries to get better at spotting the fake data.

• Variational autoencoders (VAEs)

Leverages the principles of statistical inference.
They work by encoding input data into a latent space
Decoding this latent representation to generate new data.
The introduction of a randomness factor in the encoding process allows VAEs to generate diverse yet similar data instances.

Two worthy of consideration

1. Autoregressive models => predict future data points based on previous ones
2. Normalising flow models => use a series of transformations to model complex data distributions

Examples / Use Cases OF GEN AI

Arts and entertainment:

Gen AI used to create unique pieces of art, compose music, and even generate scripts for movies.
Specialised platforms have been created that use generative algorithms to turn user-submitted images into art pieces in the style of famous painters.
Deep learning models can generate musical compositions with multiple instruments, spanning a wide range of styles and genres.

With proper prompts = > Gen AI can be used to generate films scripts, novels, poems, and virtually any kind of literature imaginable.

Technology and Communications

Gen AI is used to produce human-like text responses, making the chatbot more engaging and capable of maintaining more natural and extended conversations
Create more interactive and engaging virtual assistants.
Generate human-like text makes these virtual assistants much more sophisticated and helpful than previous generations of virtual assistant technology.

Design and Architecture

Generate design options and ideas to assist graphic designers in creating unique designs in less time.
Gen AI used by architects to generate unique and efficient floor plans based on relevant training data.

Science and Medicine

In life sciences, Gen AI is used to design novel drug candidates, cutting the discovery phases to a matter of days instead of years.

For Medical imaging, GANs used to generate synthetic brain MRI images for training AI. This is useful in scenarios where data is scarce due to privacy concerns.

E-commerce

Companies are using GANs to create hyper-realistic 3D models for advertising.
These AI-generated models can be customised to fit the desired demographic and aesthetic.
Generative algorithms are used to produce personalised marketing content, helping businesses communicate more effectively with their customers.

CHALLENGES – GEN AI

What are the Challenges faced by Organizations, companies face today in implementing Gen AI ?
Technical and Ethical concerns ?

• Data requirements
• Training Complexity
• Controlling the Output
• Ethical concerns
• Regulatory hurdles

Data requirements

Gen AI models require a significant amount of high-quality, relevant data to train
Acquiring such data can be challenging: in domains where data is scarce, sensitive, or protected such as in healthcare or finance.
Ensuring the diversity and representativeness of the data to avoid bias in the generated output can be a complex task.

Data requirements – Solutions
Use of synthetic data – artificially created data that mimics the characteristics of real data.
Data companies are specialising in generating synthetic data that can be used for AI training to preserve privacy and confidentiality.

Training complexity

Training GANs or transformer-based models is computationally intensive, time-consuming, and expensive.
Requires significant resources and expertise: barrier for smaller/new organisations

Distributed training => training process is split across multiple machines or GPUs, can help accelerate the process.
Transfer learning => pre-trained model is fine-tuned on a specific task, can reduce the training complexity and resource requirements.

Controlling the output

Controlling the output of Gen AI => might generate content that is undesirable or irrelevant.
For example, create text that is imaginary, incorrect, offensive or biased.

Improving the model’s training by providing more diverse and representative data can help manage this issue.
Implementing mechanisms to filter or check the generated content can ensure its relevance and appropriateness.

Ethical concerns

Gen AI raises several ethical concerns=> authenticity and integrity of the generated content.
Deepfakes (created by GANs) can be misused to spread misinformation or for fraudulent activities.
Generative text models can be employed to create misleading news articles or fake reviews.

Establish Robust ethical guidelines for the use of Gen AI
Digital watermarking or blockchain can help track and authenticate AI-generated content.
Teach AI to the common public can make them aware of misinformation or fraud

Regulatory hurdles

Lack of clear regulatory guidelines for the use of Gen AI.
Laws and regulations struggle to keep up, leading to uncertainties and potential legal disputes as Gen AI is rapidly growing.

Continuous dialogue and collaboration between technologists, policymakers, legal experts, and society at large are needed to shape comprehensive and effective regulatory frameworks.
Promote the responsible use of Gen AI

Gen AI – Moving Ahead

Generative AI > has become an integral part of our everyday lives.
It’s emergence within the larger field of AI represents a significant leap forward.
Power of creation to make decisions, and automate processes
Leads to applications creation that were previously unimaginable
Improve customers interactions and drive efficiencies in companies
Generating realistic images and animations for the different industries
Creating synthetic data for research and training purposes

Key words

• API
• LLM
• Prompt, Tokens
• ChatBot
• Deep Learning
• Machine Learning
• GPT (Generated Pre-trained Transformer)
• NLM
• Neural Networks