Below you can find the reviewer guidelines and responsibilities. Prior to proceeding, ensure that you have acquainted yourself with both the Submission and the Code of Conduct.

Single-blind. DMLR review process is single-blind, wherein the identities of the authors of a submission are disclosed, but the reviewers' identities are kept confidential from the authors, even following the final decision.

Review Process

  1. You are assigned as a reviewer. Upon being assigned to submission, you are expected to submit your initial reviews in one month. If you are unable to do so, contact your AE for a new deadline. At DMLR, we maintain high standards for review quality and prioritize it over rapid turnaround.
    In the event of unexpected availability issues, please promptly contact your AE. Reviews will be visible to the authors and Action Editors (AEs) as they are submitted.
  2. Reviews submitted. Following the submission of all reviews, they will be visible to other reviewers and the authors of the submission. During this phase, you are expected to actively engage in discussions with the authors until the issues you have raised for discussion have been sufficiently addressed. The authors' responses to your reviews and the ensuing discussions will also be made visible publicly only if the paper is accepted. In all instances, the identities of the reviewers remain anonymous. Also note that, throughout this process, adherence to our Code of Conduct is required.
  3. Authors-reviewer discussion ends. Following this discussion period, discuss with other reviewers and your AE to determine if the submission aligns with the Acceptance Criteria. If you are confident in your recommendation, you can communicate it to the AE before the review period concludes.
  4. Final recommendation. There are two possible outcomes: Accept or Reject. If you recommend acceptance, please provide a list of revisions you suggest for the camera-ready submission. Alternatively, if you recommend rejection, be sure to provide the authors with the grounds for this decision.

Review Format

DMLR follows the review format employed by the NeurIPS Datasets and Benchmark Track, which includes the following form questions. As you formulate your review, we recommend you make use of the NeurIPS paper checklist and ensure that your arguments are well-justified. Please be mindful that reviews for accepted papers and papers for which authors have chosen to make public will be disclosed after decisions have been rendered.
  1. Summary and contributions: Briefly summarize the paper and its contributions. This is not the place to critique the paper; the authors should generally agree with a well-written summary.
  2. Strengths: Describe the strengths of the submission, considering significance of the contribution, relevance to the broader research community, quality of the research, clarity of paper, and ethical and social implications. If the submission includes a dataset and/or benchmark, consider accessibility, accountability, and transparency or the artifact.
    3. Opportunities for improvement: Explain the limitations of this work along the same axes as above.
  3. Limitations: Have the authors adequately addressed the limitations and potential negative societal impact of their work? If not, please include constructive suggestions for improvement.
    In general, authors should be rewarded rather than punished for being up front about the limitations of their work and any potential negative societal impact. You are encouraged to think through whether any critical points are missing and provide these as feedback for the authors.
  4. Correctness: Are the claims made in the submission correct? If the submission is a dataset, it is constructed in a sound way? If it is a benchmark, are the evaluation methods and experiment design appropriate and performed correctly?
  5. Clarity: Is the paper well written?
  6. Relation to prior work: Is it clearly discussed how this work differs from previous contributions?
  7. Documentation: For datasets, is there sufficient detail on data collection and organization, availability and maintenance, and ethical and responsible use? Note that dataset submissions should include documentation and intended uses; a URL for reviewer access to the dataset; and a hosting, licensing and maintenance plan. For benchmarks, is there sufficient detail to support reproducibility?
  8. Ethical concerns: If there are ethical issues with this paper, please flag the paper for an ethics review. For guidance on when this is appropriate, please review the NeurIPS Code of Ethics.
  9. Overall: Please provide an "overall score" for this submission.
  10. Confidence: Please provide a "confidence score" for your assessment of this submission to indicate how confident you are in your evaluation.
  11. Code of conduct acknowledgment: While performing my duties as a reviewer (including writing reviews and participating in discussions), I have and will continue to abide by the DMLR code of conduct.

Acknowledgments

DMLR procedures partially follow those of TMLR and NeurIPS Datasets and Benchmark Track. Some of the sections above are adapted from those procedures.

Back to the top