Document Feature Extraction with OpenAI

Using OpenAI to Extract Key Features from CVs: A Game-Changer for Finance Professionals

As AI and GenAI make their way into finance, the world of finance is undergoing a dramatic transformation and people are starting to realize "oh, Excel is outdated".

However, one area where AI and GenAI haven't been widely deployed is in HR - why is that?

Two possible reasons are:

• HR people are great human readers, but often very far from the tech world.

• Privacy concerns

It is only fair that, as more and more CVs are being written by GenAI, we analyze them by GenAI as well.

So why are we not levelling the playing field and automate part of the CV process?

But before we get down to business, let's understand the role of HR in the application process.

Precision and expertise are non-negotiable. Sorting through a mountain of CVs is a challenging task, but it must be done. Every hiring decision is crucial. The right talent can drive success, while a misstep could have costly consequences. The process has to start again over and over until the right person is found.

Traditional methods of CV screening are reliable, but they are increasingly inadequate in keeping up with the increasing amount of candidates and the complexity of their qualifications. There is a need for innovative solutions to quickly identify top talent with the specific skills and experience required for highly specialised roles.

The traditional ATS (Applicant Tracking System) HR tools streamline the recruitment process by automating the collection, sorting, and tracking of job applications. These tools allow HR professionals to manage large volumes of resumes, screen candidates based on predefined criteria, and facilitate communication with applicants. ATS tools typically include features like resume parsing, candidate ranking, interview scheduling, and integration with job boards. They save time and improve the hiring process's overall accuracy and fairness by organizing and filtering applications efficiently.

BUT... traditional ATS systems lack features such as :

1. Limited Contextual Understanding in ATS:

   Traditional ATS tools rely on basic keyword matching, which fails to identify relevant candidates. GenAI is a game-changer. It can understand CVs and accurately identify relevant qualifications and skills.

2. Bias Reinforcement in ATS:

   ATS systems highlight certain keywords or criteria more than others, which can lead to bias. GenAI is designed to reduce this bias and make the candidate selection process more diverse and inclusive.

3. Static Rules and Slow Adaptation in ATS:

   ATS systems are slow to adapt because they need manual updates. GenAI can take a job description and compare it against CVs.

GenAI might be the solution. Advanced language models extract key features from CVs and streamline the hiring process, making it faster, more accurate and less biased. This will be the task of this article:

1. Highlighting the current state of the hiring process in Finance

2. Briefly how OpenAI GenAI works

3. The Solution an implementation in python

1. The Current State of Hiring in Finance

CV screening is a key part of the hiring process. HR professionals and hiring managers usually have to look through lots of CVs to find the right ones. This takes a lot of time. Each CV must be carefully checked for the right skills and experience.

This can lead to mistakes. Details might be missed or less suitable candidates might be chosen. Also, different reviewers can give the same CV different ratings, which makes the hiring process less consistent.

Long hiring times are also a problem. As the manual screening process takes longer, the organisation spends more money. This is not just in terms of time and resources, but also in lost productivity due to unfilled positions. Sometimes, the perfect candidate is overlooked because the hiring process is slow.

The biggest risk is hiring the wrong person for the job. If CVs are not properly assessed, the right candidates might be overlooked and unsuitable people might be hired. This can lead to poor performance, lower employee satisfaction and higher turnover.

2. How OpenAI's Language Models Work

OpenAI is a leader in developing advanced language models that understand human language.

Source: Benchmarking of Large Language Models

OpenAI's language models are good at understanding and analyzing text. This makes them useful for tasks that involve lots of text, such as CV screening.

OpenAI's language models, like GPT, are at the core of its capabilities. These models understand human language, including how it is structured and the context. They analyze text by breaking it down into parts, looking at patterns and making predictions based on the context. This means they can generate relevant responses, summarize content, and even translate languages.

For CV screening, OpenAI's models can read and interpret CVs much like a human would, but faster. They can identify the structure of a CV, differentiate between sections, and extract relevant information.

Introduction to Feature Extraction: What It Is and Why It Matters

Feature extraction is a critical process in data analysis and machine learning, where specific attributes or "features" are identified and isolated from raw data for further analysis. In the context of CV screening, feature extraction involves identifying key pieces of information from a resume, such as the candidate's skills, experience, certifications, and educational background. These features are then used to assess the suitability of a candidate for a particular role.

CV Template downloaded from the following link: Source

We then uploaded this CV to ChatGPT and asked it to extract the relevant information, without specifying any further information such as the structure of the output.

Prompt:

Extract the following relevant information from the CV.
- Candidate's skills
- Experience, 
- Languages 
- Programming Languages,
- Certifications
- Educational background.

The output is shown below.

We can immediately see that GPT-4o was able to parse the entire CV correctly and extract the information requested within the prompt. However, in order to streamline this for further processes, we need to ask ChatGPT to structure the information as JSON or XML so that we can use it later in other programs to compare it with the skills from the job offer.

To do this, we need to change the prompt a little by adding the text in bold.

Extract the following relevant information from the CV and convert it into JSON-format to process it later
- Candidate's skills
- Experience, 
- Languages 
- Programming Languages,
- Certifications
- Educational background.

This will produce the following structured and computer-interpretable result, making it super easy to process and compare with your job offer.

The importance of feature extraction in CV screening cannot be overstated. By focusing on the most relevant information, it allows recruiters to quickly and accurately assess a candidate's qualifications. In addition, feature extraction reduces the noise created by irrelevant data, ensuring that only the most relevant details are considered during the hiring process.

The Advantages of Using GenAI for Feature Extraction

The use of AI, particularly OpenAI’s language models, for feature extraction in CV screening offers several significant advantages:

1. Speed: What might take a hiring manager hours to review, an AI can do in seconds, making the initial screening process much faster.

2. Accuracy: They can consistently apply the same criteria to each CV, resulting in more accurate and fair assessments of candidates.

3. Scalability: This scalability ensures that even large volumes of CVs can be screened efficiently.

4. Objectivity: By focusing purely on the data, AI helps to ensure that candidates are assessed based on their qualifications and experience, rather than subjective impressions.

5. Customization: In finance, this means the AI can be tailored to identify and prioritise the qualifications and experience most relevant to finance roles, ensuring the extracted characteristics are aligned with the unique needs of the industry.

In summary, OpenAI's advanced language models provide a powerful tool to revolutionise the CV screening process in the finance industry. By automating feature extraction, these models not only streamline the hiring process, but also improve the accuracy and relevance of candidate assessments, leading to better hiring outcomes.

3. Implementing GenAI-Driven CV Feature Extraction with OpenAI in Python

Now the cherry on top, instead of manually uploading each CV to ChatGPT and rewriting the same prompt over and over again, we will automate the whole process using Python and a connection to OpenAI.

We'll explore how to use OpenAI's GPT-based models to extract key information from CVs using Python.

This practical guide will show you how to use OpenAI's API to process CVs, extract relevant features such as skills, experience and education, and structure this information for further analysis.

Setting Up the Environment

Before we begin, ensure you have the following prerequisites:

1. Python installed on your system (Python 3.9 or later).

2. OpenAI API Key – you need to sign up for access to the OpenAI API.

3. Python Libraries (OpenAI, …)

If this is all new to you, check out my video on Youtube where I explain how to create a web application in 15 minutes using OpenAI & Streamlit to ask questions about any document.

This video will show you how to set up an OpenAI account and will also show you another very interesting application of using OpenAI and documents in finance.

Sample Code for CV Feature Extraction

Here's a basic implementation of how to use OpenAI to extract information from a CV in a structure, as shown in the example before: https://github.com/vashAI/CV_Feature_Extractor

On Github you will find two files:

• Python file: This is the exact streamlit application as seen in the video below. Just enter your OpenAI API key, download streamlit and you are ready to go. For those who need help with these steps, check out my video above.

• Notebook file: The same story as before, add your OpenAI key and you are good to go.

Potential next steps

The next steps could involve matching CVs against a job description to rank candidates. This process helps identify the most suitable applicants based on how closely their skills and experience align with the job requirements. By analyzing and scoring CVs, we can streamline the hiring process even more. If this approach interests you, a detailed analysis can be explored in a future article, providing insights on methodologies, tools, and best practices for effective CV ranking and matching. This can further optimize recruitment strategies and ensure a better fit for the role.

We are at the end…

As AI and GenAI transform finance, HR remains an area where adoption lags, possibly due to HR professionals' distance from technology and privacy concerns. However, with the rise of AI-generated CVs, it's only logical to use GenAI for analytics to level the playing field. Traditional ATS systems, while effective, struggle with the volume and complexity of modern applications. GenAI offers a solution by improving the speed, accuracy and fairness of CV screening.

By integrating these tools, HR processes can be revolutionized, ensuring better hires and a more efficient recruitment process. The future of recruitment is increasingly AI-driven.

Let’s Keep the Conversation Going!

1. Like this story

2. What are your thoughts on this article? Have any questions or thoughts? Drop a comment below!

3. Found something interesting? Highlight key points to come back to them later.

4. Want more? Check out my Substack articles for in-depth discussions and updates — it is free!

5. Catch me on YouTube: Want practical AI and Python tutorials? Subscribe to my channel for quick, project-based videos.

6. Dive deeper into AI and Finance? Join me at the AI Finance Club to explore new trends and strategies.