A.I. Content Coder: Documentation

A.I.-assisted text analysis of open-ended survey responses. Use MQO’s fine-tuned model to develop a Codebook of common themes with example quotes. Optionally code any number of individual responses with their themes, ready for respondent-level export.

File Upload

[click here to view upload file formatting details]

Can accept short or long responses. Upload will clip any individual response to a maximum of 5,000 characters (which is really quite long).
Can accept any number of responses. (rows)
- Note that Qoken usage is influenced by overall number of responses, and average character counts across the responses.
- The “A.I. Generate a Codebook” step (producing the themes from the responses file) uses much fewer Qokens than the “Perform A.I. Coding” step (tagging individual responses with their theme categories, for respondent-level data export), especially with larger files.
The order of the responses in the uploaded file is automatically randomized upon upload. You can download a copy of the responses in the randomized order used in your session by clicking the button after upload:

Generating the Codebook

A Codebook is the list of themes identified from your open-ended responses.

“A.I. Generate the Codebook” button:

Produces a pop-up, including:

Question Text: The wording of the survey question your participants responded to. Automatically populates the text contained in cell B1 of the uploaded file. You can edit this field.
Additional Context (optional): Provide any relevant context, such as the purpose of the survey, target population, or any specific aspect you want A.I. to consider while identifying themes. Not required.
Click “Generate Codebook” to begin the A.I. process. (Takes a minute or two)

“Detailed Codebook” field:

The themes identified by the A.I. in your responses, including descriptions of what they mean in context and example respondent quotes for each theme. You can Copy (to clipboard) or Download (txt file) the Detailed Codebook using the buttons at the top of the field.

To directly edit the Detailed Codebook once it is generated, use the Edit Codebook button.

(Tip: if there are some example responses given that you don’t think are a good representation of a category, you can remove or change them here. This will help fine tune the A.I.’s performance in the A.I. Content Coding step and can be especially useful in helping it understand ‘none’ or ‘nothing’ responses in cases where these categories may be ambiguous)

To provide feedback to the A.I. on its generated Codebook, like merging/separating categories or telling the A.I. that it made a mistake, use the A.I. Refine button to start a chat.

(Tip: you may want to use the Save This Configuration button to get back to your original before refining to be able to return to it any time if you prefer)

“Re-Try” button:

If the A.I. missed the mark on its first attempt at generating categories, you can use the Re-Try button to have the A.I. make another attempt using the same inputs and settings. Some ‘creativity’ is built into the model, and simply getting the A.I. to try again can often get a great result even if it fell short the first time.

“Clean Codebook” field:

The same list of theme categories as in the Detailed Codebook, but not including the detailed descriptions and example quotes. You can directly edit the Clean Codebook, or edit the Detailed Codebook. The Clean Codebook is automatically updated when the Detailed Codebook is edited. You can Copy (to clipboard) or Download (txt file) the Clean Codebook using the buttons at the top of the field.

You can also paste in your own numbered code categories here to perform Content Code with a pre-existing Codebook.

“Add DK/NA” and “Add Other” buttons:

We highly recommend adding categories to the end of your Codebook for “DK/NA” (Don’t know / No answer) and “Other” (Miscellaneous / Uncategorized) if these categories are not present within the A.I. Generated Codebook (often, they’re not). This is optional and you can feel free to change the wording of these categories, but we do strongly recommend including them as options.

Performing Respondent-Level Content Coding

In this step, individual responses are ‘tagged’ with the numeric value from the Codebook associated with the theme category (or categories, if more than 1 allowed — see below) contained within each response. This output is formatted as comma-separated values, as:

[Respondent ID], [Category #]

For example if the response from RespondentID 547 was coded as associated with theme category #3 and RespondentID 281 with theme category #1, the output for these two records would be:

547, 3
281, 1

And so on. You can Copy (to clipboard) or Download (csv file) the coded responses using the buttons at the top of the field once complete.

“Max Number of Codes per Response” selector:

Can be set to 1, 2, or 3: the maximum number of theme categories you want to allow the A.I. to consider attributing to any individual response. If 2 or 3 are selected, they will be added to the response as an additional column in comma-separated format, as:

[Respondent ID], [First Category], [Second Category], [Third Category]

For example if Respondent 281 touched not only on theme category #1 but also theme category #7 (while Respondent 547 only mentioned theme category #3), the output for these two records would be:

547, 3
281, 1, 7

“A.I. Test Coding” button:

Performs Content Coding on only the first 150 responses (from the Randomly Ordered Responses discussed above). We highly recommend performing human review of the A.I. performance by manually coding 75 to 100 (up to 150 if the whole file is very large and/or your Codebook is quite long).

Note: Your Organization Owner was provided with a Human-A.I. Verification Checker tool and usage guidelines to assist you with this, please reach out to them or us if you need a copy.

If your file is large, you can use the A.I. Test Coding button to perform Content Coding on only 150 cases for you to verify before executing Content Coding on all responses in the uploaded file to reduce your Qoken usage.

“Perform A.I. Coding” button:

Proceeds to perform Content Coding for all uploaded responses. Large files are split into batches in the background and analyzed in parallel. You will see the first 75 coded responses stream in the Output field. The remaining coded responses will be appended in their correct order (per the Randomly Ordered Responses discussed above) once all parallel batches have completed their coding operations in the background.

Saving Configurations:

The “Save this configuration” button will save a snapshot of the current state of your session, including the uploaded file, Codebooks, and coded respondent-level output, if populated. Select “View Saved Configurations” to retrieve a previously saved configuration.