Creating Quizzes with QuestionObject

Bob Sandheinrich, Wolfram

(RowDefault[, (Spacer1), , (Spacer1), ])  #WolframTechConf

ABSTRACT

The question and assessment framework in Wolfram Language continues to advance. In this talk, I demonstrate the newest QuestionObject interfaces including fill-in-the-blank questions and dynamic sorting interfaces all supporting advanced automated assessments. I also introduce the new question notebook system for creating full tests and quizzes.

Question Framework Basics

The first half of the talk focuses on QuestionInterface.

QuestionObject[
QuestionInterface[
"Interface Type",
<|"InterfaceSettings"->xxxx|>
]
,
AssessmentFunction[
answerkey,
assessmentsettings
]
]

Randomized questions are automated with QuestionGenerator:

QuestionGenerator[
{"param1":>val1,...},
QuestionObject[
f[#param1]
]&
]

New Question Interfaces

New interfaces in 14.1:

TextCompletion

SelectCompletion

DragCompletion

DragCategorize

Sort

See the full list.

TextCompletion

A simple example

simpletextquestion=QuestionObject[QuestionInterface["TextCompletion",<|"Prompt"->"Fill in the countries","Template"->"Shanghai is in `` and London is in ``"|>],AssessmentFunction[{{"China","England"}}]]

A more interesting example

historyquestions=QuestionGenerator[{"rand":>RandomEntity[FilteredEntityClass["HistoricalCountry",(EntityProperty["has polygon?", EntityProperty["HistoricalCountry","HasPolygon"], "EntityProperty[\"HistoricalCountry\", \"HasPolygon\"]"])]]},QuestionObject[QuestionInterface["TextCompletion",<|"Prompt"->Style["Complete the sentence using a modern day country, city, landmark, etc.",Small],"Template"->"``"<>StringTemplate[" is located entirely within what was ``"][#rand],"Interpreter"->"Location"|>],With[{poly=#rand[(EntityProperty["polygon", EntityProperty["HistoricalCountry","Polygon"], "EntityProperty[\"HistoricalCountry\", \"Polygon\"]"])]},
AssessmentFunction[{{_?(GeoWithinQ[poly,#]&)}}]]]&]

historyquestions[]

Change to “SelectCompletion”

QuestionGenerator[{"rand":>RandomEntity[FilteredEntityClass["HistoricalCountry",(EntityProperty["has polygon?", EntityProperty["HistoricalCountry","HasPolygon"], "EntityProperty[\"HistoricalCountry\", \"HasPolygon\"]"])]]},&][]

Change to “DragCompletion”

QuestionGenerator[{"rand":>RandomEntity[FilteredEntityClass["HistoricalCountry",(EntityProperty["has polygon?", EntityProperty["HistoricalCountry","HasPolygon"], "EntityProperty[\"HistoricalCountry\", \"HasPolygon\"]"])]]},&][]

DragCategorize

A simple example

dragquestion=QuestionObject[QuestionInterface["DragCategorize","Categorize the primates"],
AssessmentFunction[{
"Chimpanzee"-><|"Category"->"Ape"|>,
"Gibbon"-><|"Category"->"Ape"|>,
"Baboon"-><|"Category"->"Monkey"|>,
"Sifaka"-><|"Category"->"Lemur"|>},
<|"ListAssessment"->"SeparatelyScoreElements"|>]]

A more interesting example

chemtest=QuestionGenerator[{"elements":>RandomSample[EntityList@EntityClass["Element","Nonmetal"],5]},QuestionObject[QuestionInterface["DragCategorize",<|"Prompt"->"Classify the chemical elements in their corresponding groups",
"AllowDefaultAnswers"->False|>],AssessmentFunction[MapThread[#1-><|"Category"->#2|>&,{EntityList@#elements,EntityValue[#elements,"Series"]}],
<|"ListAssessment"->"SeparatelyScoreElements"|>]]&]

chemtest[]

One Math Question In Many Forms

Expand 6x*(3x^2+a)?

Very simple:

simpletextmath=QuestionObject[
QuestionInterface["TextCompletion", 
<|"Template" -> "``x^``+``a x", "Interpreter" -> "Integer", 
"Prompt"->"Expand 6x*(3x^2+a)?"
|>], 
  AssessmentFunction[{{18, 3, 6}}]]

A nicer looking interface:

QuestionObject[
QuestionInterface["TextCompletion",
<|"Template"->TemplateObject[
{TemplateSlot[1],Superscript["x",TemplateSlot[2]],"+", TemplateSlot[3],"a*x"}],"Interpreter"->"Integer",
"Prompt"->StringForm["Expand ``?",HoldForm[6x*(3x^2+a)]]|>],
AssessmentFunction[{{18, 3, 6}}]
]

With drop downs:

QuestionObject[
QuestionInterface["SelectCompletion",
<|"Template"->TemplateObject[
{TemplateSlot[1],Superscript["x",TemplateSlot[2]],"+" TemplateSlot[3],"a x"}],"Interpreter"->"Integer",
"Prompt"->StringForm["Expand ``?",HoldForm[6x*(3x^2+a)]]|>],
AssessmentFunction[{{18,3,6}->1,{12,2,6},{8,9,1},{9,11,10}}]
]

Draggable interface:

QuestionObject[
QuestionInterface["DragCompletion",
<|"Template"->TemplateObject[
{TemplateSlot[1],Superscript["x",TemplateSlot[2]],"+" TemplateSlot[3],"a x"}],"Interpreter"->"Integer",
"Prompt"->StringForm["Expand ``?",HoldForm[6x*(3x^2+a)]]|>],
AssessmentFunction[{{18,3,6}->1,{12,2,6},{8,9,1},{9,11,10}}]
]

With automated math interpretation:

QuestionObject[
QuestionInterface["ShortAnswer",
<|"Interpreter"->"HeldMathExpression","Prompt"->StringForm["Expand ``?",HoldForm[6x*(3x^2+a)]]|>],
AssessmentFunction[Hold[6 a x+18 (x^3)],"ArithmeticResult"]
]

With an input-like field:

mathinput=QuestionObject[
QuestionInterface["ShortAnswer",
<|"FieldType"->Hold[Expression],"Prompt"->"Expand 6x*(3x^2+a)?"|>],
AssessmentFunction[Hold[6 a x+18 (x^3)],"ArithmeticResult"]
]

Specialized Polynomial assessment:

QuestionObject[
QuestionInterface["ShortAnswer",
<|"FieldType"->Hold[Expression],"Prompt"->StringForm["Expand ``?",HoldForm[6x*(3x^2+a)]]|>],
AssessmentFunction[Hold[6 a x+18 (x^3)],"PolynomialResult"]
]

A stupid question:

QuestionObject[QuestionInterface["ClickLocations",
<|"Background"->Rasterize[Style[Grid@Partition[Range[30],6],60,FontFamily->"Source Sans Pro"],ImageSize->Medium],
"Prompt"->StringForm["Prompt click the missing values ``",HoldForm[6x*(3x^2+a)==6 a x+__ (x^3)]]|>],
AssessmentFunction[{{{891,383}}->1},<|"ComparisonMethod"->"Vector","ListAssessment"->"AllElementsOrderless"|>,Tolerance->80]
]

New Question Notebook

In 14.1 the Question Notebook system was rebuilt. The major difference is a dedicated editing notebook for each question.

CreateNotebook["QuestionNotebook"]

QuestionFramework`CreateQuestionNotebook[{simpletextmath,mathinput,simpletextquestion, dragquestion}]

Question Framework Applications

CBM: Computer Based Math Educational curriculum and software

WELE: full education solutions

Wolfram Tutor: an upcoming AI-based tutoring platform

Other Talks

Advanced Automated Assessment for Math and More

Wolfram Consulting AI Math Assistant and Enabling Components

JerryBot: A Proactive AI Tutor for Algebra One

Converting OpenStax Textbooks for Use by LLMs, with an Example Application to Chemistry Education

Thank you

Questions

To be answered manually, by me, a human.