CALL FOR CHALLENGES

The IEEE International Symposium on Biomedical Imaging (ISBI) 2023 is soliciting proposals for scientific challenges. Challenges have become an intrinsic part of ISBI in the past few years. The aim of the challenges is to accelerate the pace of biomedical science on demanding research and clinical problems. To achieve this goal, a challenge should aid quantitative comparisons of competing approaches to cutting-edge research problems in biomedical image analysis, using standardized datasets, evaluation metrics, and multi-institutional collaborations.

Each challenge proposal needs an organizer who will be responsible for providing training and testing datasets, defining the tasks, specifying the performance metrics, managing and evaluating entries (through appropriate software environments), and organizing on-site presentations in the challenge workshop. The organizers are ultimately responsible for the success of the challenge. ISBI will assist in advertising the challenges.

The organization of a challenge and the agenda for each workshop will be determined by the challenge organizers. The challenge organizers should not rely solely on the on-site evaluation. We encourage scientific presentations at the workshop by the challenge participants. Multi-institutional collaborations are encouraged to ensure the wider applicability of the solutions. New challenge topics as well as topics that were addressed during previous challenges are acceptable. New challenge topics could introduce new imaging devices, new biomedical applications, or existing applications that would benefit from focused attention from the biomedical imaging research community. Challenges that address previous challenge topics could feature, for example, repeating a challenge to track how the field has advanced, addressing bottlenecks in existing processing pipelines, processing larger and more heterogeneous datasets or analyzing specific sub-populations. . For both new and previous challenge topics, we encourage organizers to consider computational cost (both training and at test time) and access to high-end computing resources as factors in challenge design and evaluation of methods. We also encourage the investigation of alternative formats for hosting grand challenges.

Prospective challenge organizers are encouraged to discuss their plans with the challenge co-chairs at an early stage before the acceptance of the proposal. Challenge organizers will have the opportunity to present the summaries of their challenges during the main conference.

How to Submit a Challenge Proposal

Challenge proposals must be submitted via email to the challenge chairs as a single PDF file. Proposals should not exceed four A4-sized pages excluding references and biosketches as per the guidelines below

Please include the following in the PDF document in IEEE Transactions format:

1. Header: Include challenge title, abstract, and keywords (up to five), as well as the name, affiliation, and contact information of the organizers.
2. Abstract
3. Introduction: Include the description of and motivation for the problem being addressed by the challenge, including the clinical context.
4. Data and Challenge: Describe the dataset that will be used, and how it will be allocated for training and testing (with approximate number of samples), including the format of the training target and desired output. If there are differences in the sources or sub-populations of the training and the testing data, then the motivation for the differences should be described (see #5 below). Also, include a description of the challenge tasks. For example, the challenge task could be segmentation where the segmentation map has to be submitted in a specific format.
5. Baseline and Evaluation: Include the details of the reference standard, such as the organizers’ basic or a previously published solution, and evaluation metrics. We encourage the contest organizers to, either themselves or through the challenge participants, test the robustness of the results. This may include, but need not be limited to, factoring in variance in results on various bootstrapped samples or testing on external cohorts with imaging protocols, anatomical protocols, or population characteristics, that did not contribute to the training cohort. Additionally, we encourage challenge organizers to mention which portions of the BIAS checklist were relevant and considered.[https://arxiv.org/ftp/arxiv/papers/1910/1910.04071.pdf]
6. Plan and schedule: A plan for organizing the contest should include the schedule for the release of the following: training data, algorithm or code for metric computation, single or multiple phases of test data, format and dummy examples of the submission files, format of the participant workshop manuscripts, submission of test results, submission of the manuscripts, declaration of the leaderboard, presentations at the workshop, and post-workshop leaderboard release (if any). Please also clarify if the participants are expected to release their source code, and if so, under what license and in what format or on which platform. An estimate of the number of participants based on the difficulty of the challenge and previous comparable events may also be included.
7. References
8. One-paragraph biosketch of each of the organizers. Organizational roles in prior challenges should be highlighted.

IMPORTANT DATES

• Submission Deadline: 27 October 2022 (11:59pm Pacific Time)
• Accept/Reject Notification: 17 November 2022
• Challenge Website Running:
• Challenge Days:

CHALLENGE CHAIRS