Home
"Proposals" will be a web application for accepting proposals for scientific meetings, such as workshops or conferences, and facilitating the peer-review and selection process at BIRS. Since the organization already uses Ruby on Rails for it's conference management application, Workshops, RoR is a good choice of framework to develop this application with. The application should run in a Docker container.
The Guidelines for Proposal Submission may be helpful for getting an idea of what the application needs to accomplish. Although, this new app is meant to be a major improvement over our existing process.
The application should have the following features:
-
A login / user account system that connects to an identify service provider, such as Keycloak.
-
Registration of new users
-
Basic contact management for users of the system, storing names, affiliations, email addresses, and subject areas of expertise, and membership in under-represented groups in STEM, for Equity, Diversity and Inclusiveness (EDI) requirements.
-
Role-based access controls throughout the application for various types of users: admins, staff, proposal submitters, proposal reviewers, committee members.
-
User-defined committees: staff can add people to various committees, and assign permissions to committees that apply to members of that committee. There will be overlapping permissions for people who are members of multiple committees. Committees can have sub-committees, and the sub-committees inherit the permissions of their parent committee.
-
Several users can submit a single proposal (collaborators). One of them is assigned the "Contact Organizer" role, 2-4 others are supporting Organizers.
The core functionality of the application is to accept proposal submissions from Internet users. This involves:
-
Login or registration, as per above.
-
Proposal submitters must select the type of proposal they are submitting. i.e. proposal for a 2-Day Workshop, a 5-Day Workshop, a Summer School, a Research in Teams group, or a Focussed Research Group.
-
Each type of proposal will have it's own unique submission form, with different fields.
-
Each submitted proposal gets assigned a unique code, in the format [2-digit year][event type code][3-digit-integer]. For example, the first submitted 5-Day Workshop for 2023 would have the code 23w5001. "23" is the year, "w5" is for 5-Day Workshops, and "001" is unique to this proposal.
-
Submission forms will have some mandatory and some optional fields, configurable by staff.
-
Staff users should be able to specify the fields on each form, whether they are mandatory, and the viewing privileges of committee members for each field. e.g. members of the "Foo" committee can see all fields of 5 Day Workshop proposals, but members of the "Bar" committee can see only fields A, B, C, F and G.
-
Text fields should have an editor that supports Markdown and LaTeX. The LaTeX editor could be embedded in the web form, like TeXStudio, or it may have to run as a separate service, such as CoCalc.
-
Submitters should be able to view the rendered version of their LaTeX document (i.e. in PDF).
-
Part of a proposal submission form is a "Potential Participants" field, where we ask submitters to provide a list of people who should be invited to attend the event, should the proposal be accepted. We ask for names, affiliations, whether they have expressed interest in attending already, and whether they are members of under-represented groups in STEM, for EDI reviews, including which under-represented group from the government list of under-represented groups.
-
The ability to upload file attachments, such as images that might be referenced in the LaTeX document.
-
Collaborative editing, so that several people submitting a single proposal can work on the document together. For this, integrating something like Overleaf would be appropriate.
-
Calling up previous proposal submissions as a template for a new submission. Proposals previously submitted by a user, but were rejected after peer-review could be called up, and feedback from review committees could be viewed. Then a "resubmit" button would open the submission form pre-filled out with the data from the old proposal.
-
A feature to help submitters with the potential participants list: they enter names and emails, and the app attempts to match with records in the BIRS database, and sends an email to them, soliciting their interest in attending the proposed event. If they are interested, it could further ask them to fill out a survey form to determine their membership in EDI groups.
-
Proposal reviewers should be presented with a list of submitted proposals that they can review. Some committee members can see all proposals, and others only the proposals assigned to them.
-
Proposals should be sortable by event code (each submission is assigned a unique code), title, or subject code. There should also be a toggle to hide all but assigned proposals. Sorting and viewing options should be remembered (cookies?).
-
The proposal list should indicate which proposals they've already reviewed, and which ones have reviews in progress but are not marked as complete.
-
Some proposal reviewers prefer doing reviews in printed format, so an easy way to compile a PDF of individual, all, selected, or assigned proposals should be provided.
-
Proposals should be presented online in the same order that they are printed in the book of proposals. For reviewers who have proposals assigned to them, a section/option that contains only the assigned proposals should be available.
-
There should be prominent “review” buttons or links in the list of proposals, so that reviewers can immediately open the review form for that proposal.
-
The review form for scientific reviews should have a grading feature, so that the reviewer can rank the strength of the proposal on a scale of 1 to 5. It should also have a text area (markdown editor) for the reviewer to leave comments on the proposal.
-
The review form for EDI reviews needs 3 grading sections, and 3 text areas.
-
The review form should save the text entered into the text fields automatically, in case the user accidentally leaves the page before saving, or their computer crashes, etc..
-
There should be a "switch" to indicate review in progress / review complete.
-
In the review form interface, there should be a “Go to Next Assigned Proposal” button, to bring up the next assigned proposal for review. We could also show a list of assigned proposals beside or below the review form, so that they could select the proposal to review next, or perhaps they just want to see a previously reviewed proposal.
-
There are at least 2 types of review: reviews for scientific merit, and reviews for EDI considerations. The review form presented depends on the user's committee membership. EDI reviewers get a form for reviewing EDI merits, and scientific reviewers get a form for reviewing scientific merits. Some users are members of multiple committees, so should be presented with both forms.
-
Proposals should be presented to reviewers in a way that is tailored to the user's committee membership. For example, members of an "EDI review" committee are interested mainly in the EDI characteristics of the proposal, so fields of the proposal that are relevant to EDI should be most prominent, with statistical summaries of the information. i.e., number of women in the proposed event, number of visible minorities. This should be consistent in printed and online versions.
-
Embedded LaTeX editor in the review field, so that the reviewer can use mathematical expressions in their review.
-
staff should have the ability to send a special link with a code in the URI to a reviewer, that grants the reviewer access to the review system to see their assigned proposals, without having to setup an account.
The Program Committees are groups of users who evaluate the reviews, to determine which proposals are to be accepted or rejected.
-
Program Committee members should be able to register and login.
-
They should see lists of proposals, and have access to the reviews of proposals. An average scientific grade and average EDI grade for each proposal should be prominent.
-
Users can be on both review committees and program committees. (Question: If this is the case, should their own grades be excluded from the averages?)
-
Some people are on multiple review committees, for example committees that are assigned to evaluate EDI reviews and committees assigned to evaluate scientific reviews. For reviewers who are members of multiple committees and may evaluate reviews of both types, the review evaluation form should have be a clear delineation between the types of reviews that they are evaluating. Also note that one reviewer may be on multiple review committees, so could leave more than one type of review.
-
Committee members should be able to leave feedback for proposal submitters whose proposals are rejected, to help them improve their proposals for the next competition.
-
There should be a printing (to PDF) feature to compile a book of proposal reviews. The order of the printed proposal reviews should be consistent with the printed book of proposals.
-
Program committee members should be able to leave notes on proposals, and on reviews.
-
Program committee members should be able to rank proposals, from 1 to the total number of proposal submissions.
-
Program committee members should be able to mark proposals as rejected, accepted, or maybe.
-
Whether these annotations, rankings, and accepted/rejected states are viewable by other members, or the Director only, or if all but the accepted/rejected status should be private, should be considered.
-
An interface for
-
The open-source collaboration platform Mattermost is an alternative to Slack. It could be setup as official BIRS groupware, and integrated into both Workshops and Proposals. With the help of separate identity provider software, we could have single-sign-on, so that one account could be used for all BIRS’ services.
