Просмотр кода страницы Development:Gradebook 2.x architecture

{{Infobox Project
|name = Gradebook 2.x architecture
|state = Planning
|tracker = MDL-25423
|discussion = [http://moodle.org/mod/forum/discuss.php?d=174346 link]
|assignee = unassigned
}}

This page summarizes some initial ideas and suggestions that may help to resolve [[Development:Gradebook improvements#Stage 3|Stage 3]] of the Gradebook improvements project.

== Current issues ==

These are some of the most important issues with the current Gradebook design that this spec tries to deal with. See http://www.mindmeister.com/89537697/gradebook-2-x-issues-braindump for the full record of the braindumping session.

; The grades appear in the gradebook immediately (MDL-25439) : The grade values are stateless. We need a way how to store grades in the gradebook but exclude them from aggregations yet. Currently, the ''hiddenuntil'' feature tries to solve this but it is pretty limited and hacky (MDL-25440).
; Students and teachers may see different values : Because of how grades hiding is implemented currently, it is practically impossible to pre-populate whole grades tree and/or export reliable data (because they are dependent on the current role permissions).
; Unable to alter grades processing : There is a demand for alternate methods of grades processing (eg grades over 100% or negative grades) due to local conventions in various countries and school systems.
; Grading logic and UI implemented in activity modules : Grading tools like single point grade, scale-based grading or a new rubric-based grading should be handled by the Gradebook core with a rich set of interface toward activity modules (and eventually other plugin types).
; Manual and auto grades interaction : What should happen in this scenario has never been defined: 1. Student makes first attempt at the quiz, gets grade of 50%, this is pushed to the gradebook. 2. Teacher manually edits the grade to be 60%. 3. Student makes second quiz attempt and scores 75%. What should the gradebook show now? -- Tim Hunt
:: a similar situation happens in AMOS when the original English text is modified after it has been translated. If we record the timestamps, we can highlight such overridden grade as "outdated" (with a possibility to mark it as up-to-date hence un-highlight it). Still the manually edited value 60% should be the valid one IMHO --[[User:David Mudrak|David Mudrak]] 15:45, 14 April 2011 (WST)

== Suggested solutions and improvements ==

The goal is to make the gradebook so that '''simple things are easy and complex things are possible'''. That is for courses that just need to maintain a list of graded activities and calculate the final grade, the setup for teacher should be straightforward using default values. Advanced usage with a complex tree of grade items and calculations happening at several layers should be possible, though the setup may require more experience and insight into the internals.

According to this document, the gradebook to be still represented as a tree of grade items. There is no need to change this. Grade items may be associated with an activity module (multiple grade items per mod are supported) or a gradebook category. Grades are aggregated from the leaves of the tree to the root grade item that represents the total course grade.

[[image:gradebook-tree.png]]

In the example above, there are two grade items associated with modules in the course. These two are aggregated into a single item associated with a grades category. This category grade item is aggregated together with yet another grade item (that itself is populated via grader report) and form the course total grade.

=== Grades in core space or plugin space ===

Generally the grade values are to be stored by the gradebook in its own core tables. This will centralize and unify the storage and processing of the grades. For most modules like Assignment, Forum, Wiki, Database etc. (and eventually other plugin types like blocks if we ever decide to support grades there, too) this is enough as they do not need to keep the grades in their own tables.

This does not mean that the modules would be forced to give up advanced grades processing features. For example Quiz or Workshop modules must store a lot of additional data to calculate the final grades. So the Quiz will still hold the calculated grades for all user's attempts. But it will push the valid one (best attempt, average, maximum or whatever is set) to the gradebook. Similarly the Workshop will hold the information how the assessment forms were filled by peers. But the final grades for each student will be pushed into the gradebook tables via some nice API.

=== Grade approval process and grade values states ===

This is a suggestion to extend the grade_grades table so that every grade item can actually hold three independent values:

* ''Raw'' grade value - This is the value that comes from the source - from the activity module, from the grader report or as a result of the aggregation at lower levels. It behaves as an input buffer that just keeps the most recent incoming value.
* ''Overridden'' grade value - This is the value of the grade defined in the gradebook.
* ''Final'' grade value - This is a public value of the grade, it is used for aggregations at higher levels, it is being exported, displayed to users etc.

[[image:gradebook-buffer.png]]

The key point of this new scheme is how and when the ''final'' value is populated. The container (column) holding the final value is populated during the grade ''approval''. Whenever the gradebook tree is being recalculated, the following procedure is taken:

* If the value of overridden grade is not null, then use it as the final value (and break as we are finished)
* If there are no approval conditions defined, take the raw grade value and use it as the final value
* If there are some approval conditions defined, evaluate the list of conditions. If the evaluation passes, use the raw grade as the final grade value
* Otherwise, set the final grade to null and eventually indicate if there are some raw grade values available but on hold.

The approval conditions mechanism should support at least the following types of conditions:

* Date-based conditions - This would replace the current hidden-until feature
* Event-based conditions - Things like "take the grades into account only after all submissions are assessed"
* Manual trigger - special type of an event. The teacher can easily order the gradebook to "take the grades into account NOW" by clicking an icon or so

At the moment, we are unable to distinguish between the incoming ''raw'' grade and the outgoing ''final'' grade value. Therefore it is not easy to put a grade on hold. For teachers in schools, it is pretty natural to have some grades already known but not having them counted into the total grade yet.

Note that by default there are no approval conditions. So the raw grades becomes automatically final grades (backward compatible behaviour). So simple scenarios are kept simple while complex scenarios possible.

To be decided: do the conditions have to pass to take the overridden value into account, too? That is - if there are some approval conditions defined, is the overridden value dependent on them or not?

=== Grade values and their interpretation and display ===

Currently, grading configuration (like max grade, grade type etc) is stored by modules themselves, often independently on the associated grade item settings. This is a proposal to change it so:

* The modules themselves do not know anything about the actual grade setting. That is, they do not know max grade, min grade, the grade display type etc.
* All the grade items settings is stored and controlled exclusively by the the gradebook code
* There is a callback mechanism introduced so that the gradebook can push the required grades setting into the mod_edit form and the module settings node in the Settings block (compare with how enrol plugins push their settings into the course edit form and how they have their own setting in the course settings node)
* When pushing a grade into the gradebook (so that is saved as the ''raw'' grade value - see above), the activity module uses normalized decimal value from 0.00000 to 1.00000.
* The activity can ask the gradebook via its public API to format a given float value according the current grade item settings. "Hey gradebook, this student reached 0.81076. What shall I print to them?" and the gradebook would return "4/5" or "pretty good" or "81%" or whatever is defined there.

So the final, overridden and raw grade values are stored as normalized 0.00000 - 1.00000 decimals in grade_grades. Their actual interpretation depends on the grade item settings.

Question: should the final value depend on the actual grade item type? Example: let us have a grade item the type of which is three-points scale 1, 2, 3 and the raw grade value is 0.97654. During the approval, should the value be transformed to 1.00000 or should it stay original?

=== Introducing gradebook engines ===

The grade values stored in the tree and the aggregation and approval logic will be separated. Right now, the grade item itself contain the computation logic. A new class, so called gradebook engine, will be responsible for all grade values handling in the future. This should be pluggable and OOP-ish. This way, we should be able to implement alternative calculation engines, for example one that does not strip normalized values to <0; 1> interval but supports grades over 100% or negative grades.

=== Rubric support ===

At the moment, rubric assessment tool is supported by the workshop module only. There is a demand to use rubric as a general assessment tool across all activities. Rubric gradeitems will be probably stored separately and the gradebook/rubric engine will process them, resulting in a single 0.000000 - 1.00000 value to be pushed into the representing gradebook gradeitem. Alternatively, rubric items can be stored as ordinary gradebook gradeitems but they will be hidden from the normal gradebook report (only the result of the rubric aggregation would be displayed).

Rubric must not depend on standard scales as most rubric items will use their own scale that does not have much sense outside the context of the given rubric.

=== Grades over 100% ===

As this document suggests to separate the grades storage from the grade processing logic, there can be a separate gradebook engine designed as needed to process grades over 100%. Apparently the storage engine must support grade values > 1.00000 (and actually even grade values < 0.00000).

=== Problem with different values seen by the teacher and by students ===

The student views only show the final grades. And only final grades are exported to external systems. Teacher is able to view raw grades, too. It would be nice to have a way how the teacher could preview the final grades as they would look like if a set of raw grades were approved (dry-run approval).

== Example ==

Let us look at an example of how this mechanism could be used in a real course. Suppose there is a course called "Defence Against the Dark Arts" (D.A.D.A). The gradebook in that course is organized into three top categories (Autumn term, Spring term and Summer term), all using "Sum of grades" aggregation. The course total is aggregated as a sum, too.

This is how "Categories and items > Simple view" gradebook page looks like. Note the semaphore (traffic light) icons. The teacher can click the semaphore icon to open a page where approval conditions for the given grade item are configured. He can add more condition or remove/modify the current ones.

[[image:gradebook-buffer-ex0.png]]

The autumn term has just started and the students' current task is to attempt the preliminary quiz. They already got grades for their introduction essays. Right now, some students already have got a grade for the preliminary quiz while others are still waiting for the attempt.

The teacher has set up the grade items in the gradebook in the following way:

* There was a condition on the "Introduction essay" that the teacher had to manually approve the grades before they became available to students. He already did so and therefore the green semaphore icon is displayed at the column heading of the grader report and the final grades are displayed normally.
* There is a condition on the "Preliminary quiz" that the grades will be hidden from students until the quiz closes. Because the quiz has not closed yet, there is the red semaphore icon displayed and the grades are in parentheses (round brackets) and slightly dimmed. These are raw grades available to the teacher, students can't see them yet. Once the quiz module sends a signal via gradebook API, the raw grades will be approved and will become final grades.
* There is a date condition on the "Course total" item so that the total grade is not available to student until the end of the school year. Again, the teacher can see the raw values but students can't.
* No other grade items have approval conditions defined so no semaphore icon is displayed for them. Their raw grades becomes final grade automatically and immediately - see for example "Category total" for the Autumn term category.

[[image:gradebook-buffer-ex1.png]]

Clicking the semaphore icon in the heading of the grader report executes the dry-run switch (approval or hiding the grade item) so the teacher can see how the grades would change. Clicking the green semaphore simulates hiding the grades (so the raw grades would be displayed only), clicking the red semaphore simulates approving the grades. In both cases, a big red warning at the top of the screen "This is just a preview" or so is displayed.

Let us look closely at Hermione's grades, for instance. She got 20/20 for the introduction essay and got 10/10 for the preliminary quiz. However, she does not know the result of the quiz yet as that grade has not been approved yet. Therefore it is not aggregated into the category total. Her course total grade so far is 20/100 but again, this is not displayed to her yet.