Debug Code with Synthetic Data From LangChain & Large Language Models | Machine-Learning Model Tests

imagine you have written a very long and complicated machine learning code that will need to run over real world data to perform a task you have written the code to handle all the foreseen errors that might happen however still not sure about unforeseen errors also you don't have time to collect data under different conditions different populations for the sake of just debugging your code here is when the artificial data comes to the rescue in this video we will a simple machine learning code to predict body mass index using the height and weight of a person then we show how to generate synthetic data for the model in the first case our data will be smooth without any missing values the code will run just fine in the second attempt we ask the large language model to randomly add missing values we see that our code fails to run due to the missing values by repeating the generation of artificial data under different conditions we can very quickly debug our codes this cell defines a function called train uncore my data that trains and tests a linear regression model using the psyit learn Library train uncore my data function takes two arguments X and Yi X represents the features independent variables of the data set and Y represents the target variable dependent variable the Trainor testore split function is used to split the data set into training and testing sets it splits the data into two sets one for training the model xcore Trin and Yore Trin and the other for testing the model xcore test and Yore test here 80% of the data is used for training the fit method is called on the model object to train the linear regression model using the training data the predict method is called on the model with the Unseen test data and the performance of the model is evaluated using two metrics one mean squared error msse two R squar score R2 finally the trained linear regression model is returned from the [Music] function [Music] this cell is designed to import API keys from a abmv file located in the same directory by utilizing this approach sensitive information such as API keys can be stored securely in a separate file and accessed within the python script without hardcoding them directly enhancing security and maintainability this code defines a python class named BMI using pantic a data validation and settings management Library the bmi's class inherits from base model indicating that it will have the functionalities provided by pantic for data modeling within the class definition there are three attributes BMI height and weight each representing different aspects of body mass index BMI calculation bmis is expected to be an integer while height and weight are expected to be FL loing Point numbers this class essentially serves as a structured representation for BMI data enforcing type constraints on the attributes to ensure data integrity and consistency when instances of this class are created or manipulated this example data represents two instances of BMI data each containing values for BMI height and weight these examples will be given to our large language model to learn the trend from them and generate data with a similar logic this code sets up a template for guiding a language model to generate synthetic data in a tabular format it defines an open AI template which serves as a basic structure for generating data next it constructs a few shot prompt template which provides specific examples like example data points to guide the model in generating similar data the template includes prefixes and suffixes for formatting along with placeholder input variables such subject and extra we will use subject to tell the language model the subject of our data and we'll use the placeholder extra to tell the language model extra features that the generated data should have this cell sets up a data generator to create synthetic data using the large language model provided by open AI it initializes the generator using a function called create open eyore dataor generator this function takes several parameters output underscore schema llm and prompt the output underscore schema defines the structure of the data to be generated in this case it is the BMI class that we defined above the llm parameter specifies the language model to be used which is from open Ai and is instantiated with specific settings like temperature a parameter controlling Randomness in the model's output and an API key for authentication open iore aior key this piece of code is designed to create 10 artificial or synthetic data samples related to body mass index BMI weight and height it uses the synthetic underscore dataor generator function that we defined earlier to accomplish this task the generated data will be stored in a variable called synthetic underscore results the extra parameter is left empty which means no additional Specific Instructions or requirements are given for the data generation process and these are the artificial data the language model has created for us this cell is about taking the synthetic data generated earlier and stored in synthetic underscore results and organizing this data into a structured format using the Panda's data frame table like structure here's what happens stepbystep here we Loop through each record in the synthetic underscore results and for for each record we check if the BMI value is minus one which is used as a placeholder for missing or invalid data if BMI is minus one it appends a NN not a number to the BMI array list to denote missing data otherwise it appends the actual BMI [Music] value in this cell we extract the feature and Target data out of the data frame and feed them into the function we wrote earlier to use these data and train a linear regression model for us remember our goal was to train a linear model that given the weight and height of a person predicts its BMI so far so good since our code ran just fine and no error was returned however we need to test this machine learning code under other conditions what if there are missing values in our code in real world data we always have missing values so let's try to generate another set of synthetic data that contains missing [Music] values we now use the variable extra to tell the language model to add some missing values into the generated [Music] data let's now run all the rest of the code as [Music] [Music] before sweet we just encountered an error showing that our code cannot handle missing values we need to get back the very first cell modify our code to handle the case of missing values and rerun the training function when no further error is returned we should generate more artificial data with other conditions that resemble real data and see if our code can handle those types of data as well and this is the whole reason we are using artificial data to generate data under different conditions very fast and don't waste our time to collect data for purposes like debugging our machine learning code utilizing artificial data in machine learning processes demands careful consideration particularly in its application towards predictive models while synthetic data sets can serve as valuable tools for debugging and testing algorithms under controlled conditions they inherently lack the nuanced patterns and unpredictable variations present in real world data this discrepancy arises because artificial data is often generated based on assumptions and simplified models that cannot encapsulate the complex dynamics of natural data sets relying on such synthetic data for training machine learning models can lead to misleading outcomes as the models may not learn the essential features and correlations necessary for accurate predictions in real scenarios hence while artificial data can be a useful asset in the initial stages of model development its limitations must be acknowledged to ensure the development of robust and reliable predictive models

Channel: CompuFlair

Views: 55

Keywords:

Id: XHCiV5T6mJw

Length: 10min 0sec (600 seconds)

Published: Sat Mar 23 2024