Similar question-and-answer systems are used to automate call centers and customer support services, in chatbots and search engines. For more information about AGI-based question-answer systems, their use and construction methods, see the webinar from the developers of the task.
Below you will find a data set consisting of texts and questions for them with different answer options. Your task: write an algorithm that will determine which answers are correct.
Basic principles:
To solve the problem, you are provided with a dataset in which there are about 6,000 questions for more than 800 texts from 5 different areas:
The task is a binary classification (True/False).
For verification, you must provide the system with a jsonl file with response labels: if the answer is correct - 1, if not - 0.
The quality of the solution is defined as the average value of accuracy and completeness of answers – F1 average.
You also have access to a basic solution from the developers of the problem and webinars with its analysis: first and second.