Registration
Candidates are able to register, pay any fees and enrol for one or more tests online. Once enrolled they simply have to log in to schedule preferred dates and times (if this is an option), monitor schedules and updates, receive private or global communications, see the status of marking process and the tests they have completed and their final results. This private portal also provides access to direct Q&A, help desk and email communications.
The candidate’s profile data is uploaded into the system and aligned to each assessment and results.
Item & Paper Creation, Review & Management
The solution can interface with existing item banks or can be used directly to create, review and manage items. Papers can be automatically compiled and version control provides additional flexibility to the review process.
Extensive question banks with multiple choice, written answers, image analysis (e-Film Reading) and case scenarios are supported. Questions can be developed collaboratively and like the training and CPD modules, can leverage highly graphical, dynamic images.
Business rules and meta-data guide the selection of each set of questions with multiple variants on the extraction criteria including ratio, question type or category.
Schedule Optimisation
Genix has integrated the Opturion Event Planner and Timetabling optimisation capabilities to provide unprecedented control of exam logistics. Allocation of people to locations, tasks and shifts to match the number and expertise of the people available can be managed. Users can enter constraints such as the examination date, work balance, breaks, days off, staff preferences and other needs. Once optimised a calendar is generated that can be viewed by the team and global or private alerts can be set.
Exam Delivery
Exams can be delivered at the same time, in a highly secure hosted environment across multiple locations around the world. Multi-layer authentication protocols prevent unauthorised access and together with captive browsers (to restrict access to other applications and some keyboard functions) prevent cheating. Invigilators have full control over candidate exam sessions including pause, termination and time restrictions at the question level.
The system can automatically score and check to ensure that no question remains unmarked. This is complemented by adaptable marking workflows that can be tailored as required.
Automated Marking
The inbuilt online marking capabilities significantly increase marker performance by supporting verification, issues resolution, escalation and validation. Results are available for immediate review and human intervention can be prompted and monitored from within the system.
The solution includes text-mining algorithms designed by Genix’s partner Conscious Machines that allow structured and unstructured data to be quickly mined in one pass. This provides examiners with the opportunity to automate essay and long-form answers prior to results being reviewed by human markers. Detailed analysis of the results can also be undertaken with Genix’s advanced analytics features.
A full audit trail is provided for legal defensibility.
Results Delivery
Final results can be delivered straight to the student as soon as they become available through the online portal. No need to add postal services to the scheduling constraints.
Online Assessments Exam Delivery for the Digital Age
The Examination Management solution developed by Genix is built, like all its solutions on the iBPMS, an intelligent process automation platform that allows Genix to create web-based software with proven reporting, security, content unification, communications centralisation, advanced analytics, administration, audit and flexible integration capabilities.
The advanced modular design of the platform provides our customers with the opportunity to select only those features that are right for their business and their budget.
Genix commenced the development of its examination management suite in 2011 and has developed an integrated set of modules that manage the lifecycle of candidate enrolment, item and paper creation and management, schedule optimisation, resource management, assessment delivery, automated and online human marking, analysis and reporting.
Click to see the diagram
Candidate Registration
Candidates can register to participate in forthcoming examinations through an online web portal that provides terms, conditions, privacy statements and candidate information and captures profile details, fee payments and schedule preferences. These details are automatically stored in the system and aligned to individual assessments, papers, markers and the candidate’s results.
The module also integrates with identity management systems for bulk provisioning of users, student details and single sign-on through external systems. Once registered, unique identities and passwords can be managed from this module.
The Item Bank
The Item Bank provides a highly collaborative environment for the creation of items and examiner-driven or automatic generation of papers based on user-defined criteria. Each item can be linked to a curriculum and have both an audit and usage history. Authorised users can create testlets and tests using a manual process or the blueprint framework that allows users to select items at random based on selected criteria that include tags, question categories, question types and degrees of difficulty.
The Item Bank also provides various search routines, including basic and advanced searches. The search capability also includes filtering and the ability to save regularly used searches.
Workflows to manage the item creation, review and approval lifecycle and any comments or iterative reviews & approvals are fully automated and linked to a master schedule so that tasks and progress can be monitored to ensure the critical path is achieved.
Once created, items are initially assigned a “draft” state. As they progress through the defined review and approval stages, the state is updated accordingly. Authors can then assign reviewers and review periods and reviewers automatically receive notifications within the system to advise of the tasks in their work queue. Items can also be bulk imported from external Question Banks, through a proven transformation process.
Questions that require a multiple-choice response, written answers or case scenarios are supported. With the capacity to use graphics and other media, including medical imaging (e.g. DICOM), questions are securely stored in one place with performance statistics attached.
- Multiple choice questions (MCQs)
- Extended MCQs Short answer
- Choose any three Put items in order
- Drag and drop Key in short answer
- Calculations OSCE Film reading
- Essay-type questions
- Selecting a defined number of options
- Fill-in-the-blank Matching
- Hot spots Word spots Bubble spots
- Sequence Rating scales (Likert)
Scheduling and Timetabling
The Opturion Event Planning & Timetabling (EPT) system has been integrated with the PROTRACK Examination Management module. For human event managers, the modification of a schedule to meet new requirements and constraints (due for example to unexpected attendees or extra activities) can become a nightmare. The EPT easily handles new requirements and constraints, quickly finding the schedule that minimises disruption to the event. Moreover, the Opturion solution is interactive, enabling permitted users to change features of the schedule and run the scheduler again so as to obtain a feasible schedule that satisfies the user’s revised requirements.
Assessment Delivery
The Examination module is designed for use in secure, high-stakes, low-stakes and on-demand linear assessments. The module contains functionality to support Test Administration including the creation of assessment events and test groups and the allocation of resources such as rooms and invigilators.
In 2013, Genix delivered a world-first eFilm Reading examination for the RANZCR, enabling candidates to answer questions in an environment that reflects workplace technologies.
Exams can be delivered at the same time, in a highly secure hosted environment across multiple locations around the world. Multi-layer authentication protocols prevent unauthorised access and together with captive browsers (to restrict access to other applications and some keyboard functions) prevent cheating. Invigilators have full control over candidate exam sessions including pause, termination and time restrictions at the question level.
The system can automatically score and check to ensure that no question remains unmarked. This is complemented by adaptable marking workflows that can be tailored as required.
Automated Marking
There is a facility to automatically mark MCQs, extended MCQs and short answer questions (the latter using text mining algorithms). Essay type questions are routed to markers who access and mark the candidate responses online utilising user-friendly marking keys that are presented to the marker in the online environment.
The Marking and Scoring engine module also supports a combination of machine marking (for MCQs and other item types that lend themselves to automated marking) and human marking based on user-friendly marking plans (keys) created at the start of the marking lifecycle.
Administrators set up the marking templates and assign markers and reviewers for each examination. The limits on how many papers a marker can mark in a given timeframe are established up front and this, along with the other business rules, determines the allocation of papers among markers using the optimised scheduling platform. Use of the schedule optimisation capabilities enables the identification of appropriate pairs of markers.
The markers will select the appropriate marking keys and this will record the marks for the candidate answer. The responses can be sent to the markers either as a complete paper for a certain number of candidates or as the same question across a larger number of candidates.
Double Marking
The Administrator can set marking threshold levels by question or globally for the allowable differential between the two markers and determine the treatment of the differential if it is less than the threshold value (i.e. if the average, the higher or lower of the two marks is allocated). If the differential exceeds the threshold value, then the Administrator needs to establish the rule that determines if the two markers are given the responsibility of agreeing a mark or if the marks are sent to a third party for review. These rules are configurable as are the rules relating to additional rounds of double marking.