Similar question-and-answer systems are used to automate call centers and customer support services, in chatbots and search engines. For more information about AGI-based question-answer systems, their use and construction methods, see the webinar from the developers of the task.
Below you will find a data set consisting of texts and questions for them with different answer options. Your task: write an algorithm that will determine which answers are correct.
Basic principles:
- The answer is contained in several sentences, not just one!
- The answer is not clearly (literally) stated in the text. The full match of the answer cannot be found in the original paragraph.
- There can be any number of correct answers.
To solve the problem, you are provided with a dataset in which there are about 6,000 questions for more than 800 texts from 5 different areas:
- Primary School Texts
- News
- Literary texts
- Fairy tales
- Brief content of the series
The task is a binary classification (True/False).
For verification, you must provide the system with a jsonl file with response labels: if the answer is correct - 1, if not - 0.
The quality of the solution is defined as the average value of accuracy and completeness of answers – F1 average.
You also have access to a basic solution from the developers of the problem and webinars with its analysis: first and second.
