{"id":86770,"topic":"ai","source":"The Daily Pennsylvanian","title":"Authors can identify influential AI research by ranking their own work, Wharton study finds - The Daily Pennsylvanian","url":"https://www.thedp.com/article/2026/09/penn-artificial-intelligence-peer-review-ranking-study-wharton","url_hash":"fc7d40cecd3c146fa080a9bb62733d9c100eaa4c","author":"","summary":"Authors can identify influential AI research by ranking their own work, Wharton study finds The Daily Pennsylvanian","content":"Wharton School researchers recently found that authors’ rankings of their own artificial intelligence papers may help identify influential work overlooked by traditional peer review.\nThe study, published on Aug. 24 in Nature Computational Science, analyzed authors’ comparative rankings of their own papers among those with multiple submissions to the 2023 International Conference on Machine Learning. The team embarked upon this study to determine an additional measure for identifying high-impact AI research given that the field has grown at a rate much larger than the number of experienced peer reviewers in the same areas.\nAuthors were forced to be objective because they could not give all of their work a perfect score, and the researchers found that higher-ranked papers received more academic citations than lower-ranked papers.\nThe team included statistics and data science professors Weijie Su and Bingxin Zhao, and fourth-year Applied Mathematics and Computational Science doctoral student Buxin Su.\n“We use self-rankings to identify papers that need additional attention and guide Area Chairs toward those papers,” Su wrote in a statement.\nIn the experiment, authors with multiple submissions were asked to rank their papers by perceived scientific quality shortly after the submission deadline and before reviews were released. Reviewers then evaluated the papers without seeing the rankings.\nThe study collected rankings from 1,342 authors covering 2,592 submissions. After matching the responses with citation data, the researchers analyzed a final sample of 797 authors and 1,527 unique papers.\nPapers ranked highest by their authors received an average of twice as many citations over the following 16 months as those ranked lowest. The pattern held among both accepted and rejected papers.\nRELATED:\nOf the 22 papers in the sample that received more than 150 citations, 17 — or about 77% — were ranked first by at least one of their authors. The researchers also found that author rankings predicted future citation counts more accurately than peer-review scores.\nThe study used citations as a measure of scientific impact, though citation counts do not necessarily reflect a paper’s quality. Factors including the popularity of a research topic, the timing of a paper’s public release, and an author’s professional visibility can affect how often it is cited.\nThe researchers proposed using differences between author rankings and reviewer scores to identify papers that might require closer examination, but they did not propose using self-rankings to directly determine whether a paper should be accepted.\nUnder the system, author rankings are converted into scores that can be compared with reviewers’ initial evaluations. The 10% of papers with the largest evaluation differences are flagged for an area chair, who recommends whether it should be accepted.\nArea chairs can only see the size of the discrepancy, not whether the author-based score was higher or lower than the review score. They can then examine the paper and its reviews more closely or recruit additional reviewers.\nAccording to Su, the system is designed to reduce incentives for authors to manipulate their rankings. Placing a weaker paper first could draw additional scrutiny to the submission rather than improve its likelihood of acceptance.\nICML incorporated the approach into its 2026 review process and conducted a randomized experiment to measure whether displaying the discrepancy categories affected how area chairs handled papers.\nAmong flagged papers, area chairs who could see the categories wrote approximately 71% more comment text per paper than those who could not, according to Su. They also communicated more frequently with individual reviewers, and the team observed more subsequent discussion between reviewers and authors.\nThe experiment measured participation in the review process but did not establish whether the system led to better acceptance decisions.\n“The broader implication is that author self-ranking can help conferences direct limited reviewing attention toward cases that deserve closer scrutiny,” Su wrote.","image_url":"https://snworksceo.imgix.net/dpn/4c1367ae-d70d-4ea9-bf02-c23e5f50cc25.sized-1000x1000.jpg?w=800&h=600","lang":"en","published_at":"2026-09-21T03:25:26+00:00","fetched_at":"2026-09-21T04:15:04+00:00","status":"read","starred":0,"extract_state":"ok","summary_auto":"Wharton School researchers recently found that authors’ rankings of their own artificial intelligence papers may help identify influential work overlooked by traditional peer review. 24 in Nature Computational Science, analyzed authors’ comparative rankings of their own papers among those with multiple submissions to the 2023 International Conference on Machine Learning.","cluster_id":null,"extract_retries":0,"extract_error":null,"contract_version":"news_item.v1","format_contract_version":"news_item_formats.v1","dedup_url":"https://www.thedp.com/article/2026/09/penn-artificial-intelligence-peer-review-ranking-study-wharton","quality_profile":{"profile_version":"extraction_quality.v2","bucket":"high","confidence":0.9,"failure_kind":"none","retryable":false,"retry_after_attempts":0,"reason":"High confidence: full text extraction produced 4138 characters.","operator_guidance":{"severity":"ok","recommended_action":"trust_full_text","next_step":"Use the extracted full text as the primary article source.","operator_label":"Ready","can_retry":false,"can_use_summary":false,"diagnostics_required":false},"content_depth":{"contract_version":"content_depth.v1","category":"full_text","label":"Full text","has_full_text":true,"has_summary":true,"content_length":4138,"summary_length":373,"usable_text_length":4138,"source_field":"content"},"legacy_collapsed":false,"signals":{"extract_state":"ok","extract_error":null,"extract_retries":0,"content_length":4138,"summary_length":373}},"news_item":{"id":86770,"canonical_url":"https://www.thedp.com/article/2026/09/penn-artificial-intelligence-peer-review-ranking-study-wharton","source_url":"https://www.thedp.com/article/2026/09/penn-artificial-intelligence-peer-review-ranking-study-wharton","title":"Authors can identify influential AI research by ranking their own work, Wharton study finds - The Daily Pennsylvanian","source_name":"The Daily Pennsylvanian","author":null,"published_at":"2026-09-21T03:25:26+00:00","locale":"en","topic":"ai","tags":[],"rss_summary":"Authors can identify influential AI research by ranking their own work, Wharton study finds The Daily Pennsylvanian","full_text":"Wharton School researchers recently found that authors’ rankings of their own artificial intelligence papers may help identify influential work overlooked by traditional peer review.\nThe study, published on Aug. 24 in Nature Computational Science, analyzed authors’ comparative rankings of their own papers among those with multiple submissions to the 2023 International Conference on Machine Learning. The team embarked upon this study to determine an additional measure for identifying high-impact AI research given that the field has grown at a rate much larger than the number of experienced peer reviewers in the same areas.\nAuthors were forced to be objective because they could not give all of their work a perfect score, and the researchers found that higher-ranked papers received more academic citations than lower-ranked papers.\nThe team included statistics and data science professors Weijie Su and Bingxin Zhao, and fourth-year Applied Mathematics and Computational Science doctoral student Buxin Su.\n“We use self-rankings to identify papers that need additional attention and guide Area Chairs toward those papers,” Su wrote in a statement.\nIn the experiment, authors with multiple submissions were asked to rank their papers by perceived scientific quality shortly after the submission deadline and before reviews were released. Reviewers then evaluated the papers without seeing the rankings.\nThe study collected rankings from 1,342 authors covering 2,592 submissions. After matching the responses with citation data, the researchers analyzed a final sample of 797 authors and 1,527 unique papers.\nPapers ranked highest by their authors received an average of twice as many citations over the following 16 months as those ranked lowest. The pattern held among both accepted and rejected papers.\nRELATED:\nOf the 22 papers in the sample that received more than 150 citations, 17 — or about 77% — were ranked first by at least one of their authors. The researchers also found that author rankings predicted future citation counts more accurately than peer-review scores.\nThe study used citations as a measure of scientific impact, though citation counts do not necessarily reflect a paper’s quality. Factors including the popularity of a research topic, the timing of a paper’s public release, and an author’s professional visibility can affect how often it is cited.\nThe researchers proposed using differences between author rankings and reviewer scores to identify papers that might require closer examination, but they did not propose using self-rankings to directly determine whether a paper should be accepted.\nUnder the system, author rankings are converted into scores that can be compared with reviewers’ initial evaluations. The 10% of papers with the largest evaluation differences are flagged for an area chair, who recommends whether it should be accepted.\nArea chairs can only see the size of the discrepancy, not whether the author-based score was higher or lower than the review score. They can then examine the paper and its reviews more closely or recruit additional reviewers.\nAccording to Su, the system is designed to reduce incentives for authors to manipulate their rankings. Placing a weaker paper first could draw additional scrutiny to the submission rather than improve its likelihood of acceptance.\nICML incorporated the approach into its 2026 review process and conducted a randomized experiment to measure whether displaying the discrepancy categories affected how area chairs handled papers.\nAmong flagged papers, area chairs who could see the categories wrote approximately 71% more comment text per paper than those who could not, according to Su. They also communicated more frequently with individual reviewers, and the team observed more subsequent discussion between reviewers and authors.\nThe experiment measured participation in the review process but did not establish whether the system led to better acceptance decisions.\n“The broader implication is that author self-ranking can help conferences direct limited reviewing attention toward cases that deserve closer scrutiny,” Su wrote.","excerpt":"Wharton School researchers recently found that authors’ rankings of their own artificial intelligence papers may help identify influential work overlooked by traditional peer review. 24 in Nature Computational Science, analyzed authors’ comparative rankings of their own papers among those with multiple submissions to the 2023 International Conference on Machine Learning.","extraction":{"state":"ok","confidence":0.9,"error":null,"explanation":"High confidence: full text extraction produced 4138 characters.","diagnostics_url":"/api/diagnose?url=https%3A//www.thedp.com/article/2026/09/penn-artificial-intelligence-peer-review-ranking-study-wharton","quality_profile":{"profile_version":"extraction_quality.v2","bucket":"high","confidence":0.9,"failure_kind":"none","retryable":false,"retry_after_attempts":0,"reason":"High confidence: full text extraction produced 4138 characters.","operator_guidance":{"severity":"ok","recommended_action":"trust_full_text","next_step":"Use the extracted full text as the primary article source.","operator_label":"Ready","can_retry":false,"can_use_summary":false,"diagnostics_required":false},"content_depth":{"contract_version":"content_depth.v1","category":"full_text","label":"Full text","has_full_text":true,"has_summary":true,"content_length":4138,"summary_length":373,"usable_text_length":4138,"source_field":"content"},"legacy_collapsed":false,"signals":{"extract_state":"ok","extract_error":null,"extract_retries":0,"content_length":4138,"summary_length":373}}},"display_formats":["compact","card","full","digest_section","json"]},"daily_stack_record":{"title":"Authors can identify influential AI research by ranking their own work, Wharton study finds - The Daily Pennsylvanian","url":"https://www.thedp.com/article/2026/09/penn-artificial-intelligence-peer-review-ranking-study-wharton","summary":"Wharton School researchers recently found that authors’ rankings of their own artificial intelligence papers may help identify influential work overlooked by traditional peer review. 24 in Nature Computational Science, analyzed authors’ comparative rankings of their own papers among those with multiple submissions to the 2023 International Conference on Machine Learning.","source":"The Daily Pennsylvanian","date":"2026-09-21T03:25:26+00:00","content":"Wharton School researchers recently found that authors’ rankings of their own artificial intelligence papers may help identify influential work overlooked by traditional peer review.\nThe study, published on Aug. 24 in Nature Computational Science, analyzed authors’ comparative rankings of their own papers among those with multiple submissions to the 2023 International Conference on Machine Learning. The team embarked upon this study to determine an additional measure for identifying high-impact AI research given that the field has grown at a rate much larger than the number of experienced peer reviewers in the same areas.\nAuthors were forced to be objective because they could not give all of their work a perfect score, and the researchers found that higher-ranked papers received more academic citations than lower-ranked papers.\nThe team included statistics and data science professors Weijie Su and Bingxin Zhao, and fourth-year Applied Mathematics and Computational Science doctoral student Buxin Su.\n“We use self-rankings to identify papers that need additional attention and guide Area Chairs toward those papers,” Su wrote in a statement.\nIn the experiment, authors with multiple submissions were asked to rank their papers by perceived scientific quality shortly after the submission deadline and before reviews were released. Reviewers then evaluated the papers without seeing the rankings.\nThe study collected rankings from 1,342 authors covering 2,592 submissions. After matching the responses with citation data, the researchers analyzed a final sample of 797 authors and 1,527 unique papers.\nPapers ranked highest by their authors received an average of twice as many citations over the following 16 months as those ranked lowest. The pattern held among both accepted and rejected papers.\nRELATED:\nOf the 22 papers in the sample that received more than 150 citations, 17 — or about 77% — were ranked first by at least one of their authors. The researchers also found that author rankings predicted future citation counts more accurately than peer-review scores.\nThe study used citations as a measure of scientific impact, though citation counts do not necessarily reflect a paper’s quality. Factors including the popularity of a research topic, the timing of a paper’s public release, and an author’s professional visibility can affect how often it is cited.\nThe researchers proposed using differences between author rankings and reviewer scores to identify papers that might require closer examination, but they did not propose using self-rankings to directly determine whether a paper should be accepted.\nUnder the system, author rankings are converted into scores that can be compared with reviewers’ initial evaluations. The 10% of papers with the largest evaluation differences are flagged for an area chair, who recommends whether it should be accepted.\nArea chairs can only see the size of the discrepancy, not whether the author-based score was higher or lower than the review score. They can then examine the paper and its reviews more closely or recruit additional reviewers.\nAccording to Su, the system is designed to reduce incentives for authors to manipulate their rankings. Placing a weaker paper first could draw additional scrutiny to the submission rather than improve its likelihood of acceptance.\nICML incorporated the approach into its 2026 review process and conducted a randomized experiment to measure whether displaying the discrepancy categories affected how area chairs handled papers.\nAmong flagged papers, area chairs who could see the categories wrote approximately 71% more comment text per paper than those who could not, according to Su. They also communicated more frequently with individual reviewers, and the team observed more subsequent discussion between reviewers and authors.\nThe experiment measured participation in the review process but did not establish whether the system led to better acceptance decisions.\n“The broader implication is that author self-ranking can help conferences direct limited reviewing attention toward cases that deserve closer scrutiny,” Su wrote.","confidence":0.9,"diagnostics_url":"/api/diagnose?url=https%3A//www.thedp.com/article/2026/09/penn-artificial-intelligence-peer-review-ranking-study-wharton","quality_bucket":"high","failure_kind":"none","retryable":false,"quality_reason":"High confidence: full text extraction produced 4138 characters.","quality_profile":{"profile_version":"extraction_quality.v2","bucket":"high","confidence":0.9,"failure_kind":"none","retryable":false,"retry_after_attempts":0,"reason":"High confidence: full text extraction produced 4138 characters.","operator_guidance":{"severity":"ok","recommended_action":"trust_full_text","next_step":"Use the extracted full text as the primary article source.","operator_label":"Ready","can_retry":false,"can_use_summary":false,"diagnostics_required":false},"content_depth":{"contract_version":"content_depth.v1","category":"full_text","label":"Full text","has_full_text":true,"has_summary":true,"content_length":4138,"summary_length":373,"usable_text_length":4138,"source_field":"content"},"legacy_collapsed":false,"signals":{"extract_state":"ok","extract_error":null,"extract_retries":0,"content_length":4138,"summary_length":373}},"tags":[]},"fallback_formats":["markdown","json","html"],"actions":{"read":"/item/86770","export_markdown":"/api/items/86770/export?format=markdown","export_json":"/api/items/86770/export?format=json","diagnose":"/api/diagnose?url=https%3A//www.thedp.com/article/2026/09/penn-artificial-intelligence-peer-review-ranking-study-wharton"},"formats":{"full":{"id":86770,"title":"Authors can identify influential AI research by ranking their own work, Wharton study finds - The Daily Pennsylvanian","url":"https://www.thedp.com/article/2026/09/penn-artificial-intelligence-peer-review-ranking-study-wharton","source":"The Daily Pennsylvanian","author":null,"published_at":"2026-09-21T03:25:26+00:00","locale":"en","topic":"ai","tags":[],"excerpt":"Wharton School researchers recently found that authors’ rankings of their own artificial intelligence papers may help identify influential work overlooked by traditional peer review. 24 in Nature Computational Science, analyzed authors’ comparative rankings of their own papers among those with multiple submissions to the 2023 International Conference on Machine Learning.","full_text":"Wharton School researchers recently found that authors’ rankings of their own artificial intelligence papers may help identify influential work overlooked by traditional peer review.\nThe study, published on Aug. 24 in Nature Computational Science, analyzed authors’ comparative rankings of their own papers among those with multiple submissions to the 2023 International Conference on Machine Learning. The team embarked upon this study to determine an additional measure for identifying high-impact AI research given that the field has grown at a rate much larger than the number of experienced peer reviewers in the same areas.\nAuthors were forced to be objective because they could not give all of their work a perfect score, and the researchers found that higher-ranked papers received more academic citations than lower-ranked papers.\nThe team included statistics and data science professors Weijie Su and Bingxin Zhao, and fourth-year Applied Mathematics and Computational Science doctoral student Buxin Su.\n“We use self-rankings to identify papers that need additional attention and guide Area Chairs toward those papers,” Su wrote in a statement.\nIn the experiment, authors with multiple submissions were asked to rank their papers by perceived scientific quality shortly after the submission deadline and before reviews were released. Reviewers then evaluated the papers without seeing the rankings.\nThe study collected rankings from 1,342 authors covering 2,592 submissions. After matching the responses with citation data, the researchers analyzed a final sample of 797 authors and 1,527 unique papers.\nPapers ranked highest by their authors received an average of twice as many citations over the following 16 months as those ranked lowest. The pattern held among both accepted and rejected papers.\nRELATED:\nOf the 22 papers in the sample that received more than 150 citations, 17 — or about 77% — were ranked first by at least one of their authors. The researchers also found that author rankings predicted future citation counts more accurately than peer-review scores.\nThe study used citations as a measure of scientific impact, though citation counts do not necessarily reflect a paper’s quality. Factors including the popularity of a research topic, the timing of a paper’s public release, and an author’s professional visibility can affect how often it is cited.\nThe researchers proposed using differences between author rankings and reviewer scores to identify papers that might require closer examination, but they did not propose using self-rankings to directly determine whether a paper should be accepted.\nUnder the system, author rankings are converted into scores that can be compared with reviewers’ initial evaluations. The 10% of papers with the largest evaluation differences are flagged for an area chair, who recommends whether it should be accepted.\nArea chairs can only see the size of the discrepancy, not whether the author-based score was higher or lower than the review score. They can then examine the paper and its reviews more closely or recruit additional reviewers.\nAccording to Su, the system is designed to reduce incentives for authors to manipulate their rankings. Placing a weaker paper first could draw additional scrutiny to the submission rather than improve its likelihood of acceptance.\nICML incorporated the approach into its 2026 review process and conducted a randomized experiment to measure whether displaying the discrepancy categories affected how area chairs handled papers.\nAmong flagged papers, area chairs who could see the categories wrote approximately 71% more comment text per paper than those who could not, according to Su. They also communicated more frequently with individual reviewers, and the team observed more subsequent discussion between reviewers and authors.\nThe experiment measured participation in the review process but did not establish whether the system led to better acceptance decisions.\n“The broader implication is that author self-ranking can help conferences direct limited reviewing attention toward cases that deserve closer scrutiny,” Su wrote.","reading_time_min":3,"extraction":{"state":"ok","confidence":0.9,"error":null,"explanation":"High confidence: full text extraction produced 4138 characters.","diagnostics_url":"/api/diagnose?url=https%3A//www.thedp.com/article/2026/09/penn-artificial-intelligence-peer-review-ranking-study-wharton","quality_profile":{"profile_version":"extraction_quality.v2","bucket":"high","confidence":0.9,"failure_kind":"none","retryable":false,"retry_after_attempts":0,"reason":"High confidence: full text extraction produced 4138 characters.","operator_guidance":{"severity":"ok","recommended_action":"trust_full_text","next_step":"Use the extracted full text as the primary article source.","operator_label":"Ready","can_retry":false,"can_use_summary":false,"diagnostics_required":false},"content_depth":{"contract_version":"content_depth.v1","category":"full_text","label":"Full text","has_full_text":true,"has_summary":true,"content_length":4138,"summary_length":373,"usable_text_length":4138,"source_field":"content"},"legacy_collapsed":false,"signals":{"extract_state":"ok","extract_error":null,"extract_retries":0,"content_length":4138,"summary_length":373}}},"quality_profile":{"profile_version":"extraction_quality.v2","bucket":"high","confidence":0.9,"failure_kind":"none","retryable":false,"retry_after_attempts":0,"reason":"High confidence: full text extraction produced 4138 characters.","operator_guidance":{"severity":"ok","recommended_action":"trust_full_text","next_step":"Use the extracted full text as the primary article source.","operator_label":"Ready","can_retry":false,"can_use_summary":false,"diagnostics_required":false},"content_depth":{"contract_version":"content_depth.v1","category":"full_text","label":"Full text","has_full_text":true,"has_summary":true,"content_length":4138,"summary_length":373,"usable_text_length":4138,"source_field":"content"},"legacy_collapsed":false,"signals":{"extract_state":"ok","extract_error":null,"extract_retries":0,"content_length":4138,"summary_length":373}},"actions":{"read":"/item/86770","export_markdown":"/api/items/86770/export?format=markdown","export_json":"/api/items/86770/export?format=json","diagnose":"/api/diagnose?url=https%3A//www.thedp.com/article/2026/09/penn-artificial-intelligence-peer-review-ranking-study-wharton"}},"digest":{"id":86770,"title":"Authors can identify influential AI research by ranking their own work, Wharton study finds - The Daily Pennsylvanian","url":"https://www.thedp.com/article/2026/09/penn-artificial-intelligence-peer-review-ranking-study-wharton","source":"The Daily Pennsylvanian","topic":"ai","published_at":"2026-09-21T03:25:26+00:00","excerpt":"Wharton School researchers recently found that authors’ rankings of their own artificial intelligence papers may help identify influential work overlooked by traditional peer review. 24 in Nature Computational Science, analyzed authors’ comparative rankings of their own papers…","quality_bucket":"high","quality_reason":"High confidence: full text extraction produced 4138 characters.","reading_time_min":3,"cluster_id":null},"card":{"display_title":"Authors can identify influential AI research by ranking their own work, Wharton study finds - The Daily Pennsylvanian","subtitle":"The Daily Pennsylvanian · 2026-09-21","summary":"Wharton School researchers recently found that authors’ rankings of their own artificial intelligence papers may help identify influential work overlooked by traditional peer review. 24 in Nature Computational Science,…","badges":["quality:high"],"links":{"read":"/item/86770","original":"https://www.thedp.com/article/2026/09/penn-artificial-intelligence-peer-review-ranking-study-wharton","diagnose":"/api/diagnose?url=https%3A//www.thedp.com/article/2026/09/penn-artificial-intelligence-peer-review-ranking-study-wharton"},"quality_warning":null},"export":{"title":"Authors can identify influential AI research by ranking their own work, Wharton study finds - The Daily Pennsylvanian","url":"https://www.thedp.com/article/2026/09/penn-artificial-intelligence-peer-review-ranking-study-wharton","summary":"Wharton School researchers recently found that authors’ rankings of their own artificial intelligence papers may help identify influential work overlooked by traditional peer review. 24 in Nature Computational Science, analyzed authors’ comparative rankings of their own papers among those with multiple submissions to the 2023 International Conference on Machine Learning.","source":"The Daily Pennsylvanian","date":"2026-09-21T03:25:26+00:00","content":"Wharton School researchers recently found that authors’ rankings of their own artificial intelligence papers may help identify influential work overlooked by traditional peer review.\nThe study, published on Aug. 24 in Nature Computational Science, analyzed authors’ comparative rankings of their own papers among those with multiple submissions to the 2023 International Conference on Machine Learning. The team embarked upon this study to determine an additional measure for identifying high-impact AI research given that the field has grown at a rate much larger than the number of experienced peer reviewers in the same areas.\nAuthors were forced to be objective because they could not give all of their work a perfect score, and the researchers found that higher-ranked papers received more academic citations than lower-ranked papers.\nThe team included statistics and data science professors Weijie Su and Bingxin Zhao, and fourth-year Applied Mathematics and Computational Science doctoral student Buxin Su.\n“We use self-rankings to identify papers that need additional attention and guide Area Chairs toward those papers,” Su wrote in a statement.\nIn the experiment, authors with multiple submissions were asked to rank their papers by perceived scientific quality shortly after the submission deadline and before reviews were released. Reviewers then evaluated the papers without seeing the rankings.\nThe study collected rankings from 1,342 authors covering 2,592 submissions. After matching the responses with citation data, the researchers analyzed a final sample of 797 authors and 1,527 unique papers.\nPapers ranked highest by their authors received an average of twice as many citations over the following 16 months as those ranked lowest. The pattern held among both accepted and rejected papers.\nRELATED:\nOf the 22 papers in the sample that received more than 150 citations, 17 — or about 77% — were ranked first by at least one of their authors. The researchers also found that author rankings predicted future citation counts more accurately than peer-review scores.\nThe study used citations as a measure of scientific impact, though citation counts do not necessarily reflect a paper’s quality. Factors including the popularity of a research topic, the timing of a paper’s public release, and an author’s professional visibility can affect how often it is cited.\nThe researchers proposed using differences between author rankings and reviewer scores to identify papers that might require closer examination, but they did not propose using self-rankings to directly determine whether a paper should be accepted.\nUnder the system, author rankings are converted into scores that can be compared with reviewers’ initial evaluations. The 10% of papers with the largest evaluation differences are flagged for an area chair, who recommends whether it should be accepted.\nArea chairs can only see the size of the discrepancy, not whether the author-based score was higher or lower than the review score. They can then examine the paper and its reviews more closely or recruit additional reviewers.\nAccording to Su, the system is designed to reduce incentives for authors to manipulate their rankings. Placing a weaker paper first could draw additional scrutiny to the submission rather than improve its likelihood of acceptance.\nICML incorporated the approach into its 2026 review process and conducted a randomized experiment to measure whether displaying the discrepancy categories affected how area chairs handled papers.\nAmong flagged papers, area chairs who could see the categories wrote approximately 71% more comment text per paper than those who could not, according to Su. They also communicated more frequently with individual reviewers, and the team observed more subsequent discussion between reviewers and authors.\nThe experiment measured participation in the review process but did not establish whether the system led to better acceptance decisions.\n“The broader implication is that author self-ranking can help conferences direct limited reviewing attention toward cases that deserve closer scrutiny,” Su wrote.","confidence":0.9,"diagnostics_url":"/api/diagnose?url=https%3A//www.thedp.com/article/2026/09/penn-artificial-intelligence-peer-review-ranking-study-wharton","quality_bucket":"high","failure_kind":"none","retryable":false,"quality_reason":"High confidence: full text extraction produced 4138 characters.","quality_profile":{"profile_version":"extraction_quality.v2","bucket":"high","confidence":0.9,"failure_kind":"none","retryable":false,"retry_after_attempts":0,"reason":"High confidence: full text extraction produced 4138 characters.","operator_guidance":{"severity":"ok","recommended_action":"trust_full_text","next_step":"Use the extracted full text as the primary article source.","operator_label":"Ready","can_retry":false,"can_use_summary":false,"diagnostics_required":false},"content_depth":{"contract_version":"content_depth.v1","category":"full_text","label":"Full text","has_full_text":true,"has_summary":true,"content_length":4138,"summary_length":373,"usable_text_length":4138,"source_field":"content"},"legacy_collapsed":false,"signals":{"extract_state":"ok","extract_error":null,"extract_retries":0,"content_length":4138,"summary_length":373}},"tags":[],"format_contract_version":"news_item_formats.v1"}}}