Validating the Fine-Tuning Dataset
Essential Checkpoints for Fine-Tuning Dataset Validation
If the training dataset for fine-tuning has an incorrect JSONL format or contains fewer than 10 JSON objects, the fine-tuning process will not work.
Additionally, the model may fail to learn properly if there are duplicate data, missing required fields, or inconsistent data types.
In this lesson, we will explore the critical elements you need to validate in the dataset before starting fine-tuning.
File Format Validation
First, verify that the data file follows the JSON Lines format.
A JSONL file should contain one JSON object per line.
Open the file to ensure every line is a valid JSON format. It is also advisable to check if the file is encoded in UTF-8.
{"messages": [{"role": "system", "content": "You are a friendly travel guide chatbot."}, {"role": "user", "content": "Hello, how's the weather today?"}, {"role": "assistant", "content": "Hello! The weather is sunny and clear today."}]}
{"messages": [{"role": "system", "content": "You are a friendly travel guide chatbot."}, {"role": "user", "content": "Can you recommend some tourist spots?"}, {"role": "assistant", "content": "Of course! I recommend the Grand Canyon and Central Park."}]}
Checking Required Fields
Next, ensure that all necessary fields are present in each JSON object.
For example, in a dataset containing text data and labels, each object should have a messages field and should not contain fields with names like message (without the 's') or anything else.
{"messages": [{"role": "system", "content": "You are a friendly travel guide chatbot."}, {"role": "user", "content": "Hello, how's the weather today?"}, {"role": "assistant", "content": "Hello! The weather is sunny and clear today."}]} # 'messages' field exists
{"messages": [{"role": "system", "content": "You are a friendly travel guide chatbot."}, {"role": "user", "content": "Can you recommend some tourist spots?"}, {"role": "assistant", "content": "Of course! I recommend the Grand Canyon and Central Park."}]} # Consistency maintained
Data Type Validation
Ensure that the data type for fields is consistent. The messages field should be an array of objects, and each role and content field within the objects should be a string. Mixing data types can confuse the model. Check that every JSON object has fields with the correct data type.
{"messages": [{"role": "system", "content": "You are a friendly travel guide chatbot."}, {"role": "user", "content": "Hello, how's the weather today?"}, {"role": "assistant", "content": "Hello! The weather is sunny and clear today."}]} # 'messages' is an array of objects, 'role' and 'content' are strings
{"messages": [{"role": "system", "content": "You are a friendly travel guide chatbot."}, {"role": "user", "content": "Can you recommend some tourist spots?"}, {"role": "assistant", "content": "Of course! I recommend the Grand Canyon and Central Park."}]} # Consistency maintained
Checking for Duplicate Data
Finally, check for duplicate data. Duplicate data can introduce bias into the model's learning process, so use unique identifiers in the dataset to identify and remove duplicates as needed.
{"messages": [{"role": "system", "content": "You are a friendly travel guide chatbot."}, {"role": "user", "content": "Hello, how's the weather today?"}, {"role": "assistant", "content": "Hello! The weather is sunny and clear today."}]} # Non-duplicate data
{"messages": [{"role": "system", "content": "You are a friendly travel guide chatbot."}, {"role": "user", "content": "Can you recommend some tourist spots?"}, {"role": "assistant", "content": "Of course! I recommend the Grand Canyon and Central Park."}]} # Unique data
{"messages": [{"role": "system", "content": "You are a friendly travel guide chatbot."}, {"role": "user", "content": "Hello, how's the weather today?"}, {"role": "assistant", "content": "Hello! The weather is sunny and clear today."}]} # Duplicate data, removal needed
Want to learn more?
Join CodeFriends Plus membership or enroll in a course to start your journey.