Research 🔬
Game Machine and Algorithm towards Trends in Game States using Machine Learning and Deep Learning
Published at: 2023 10th International Conference on Computing for Sustainable Global Development (INDIACom)(2023)
- Explored a hybrid approach combining machine learning and deep learning to analyze dynamic game states across various game genres.
- Applied Markov Chain models to model game environments and presented detailed use cases in popular games such as cricket, poker, chess, and football.
- Developed an in-depth hybrid algorithm specifically tailored for game-state analysis in chess.
PeerGauge: a Dataset for Peer Review Disagreement and Severity Gauge
Published at: Language Resources and Evaluation, Springer(2025)
- In this paper we proposed a novel dataset named as PeerGauge, to estimate the severity of contradictions among reviewers. This dataset provides a new dimension to understanding the degree of disagreement in peer review processes.
- Additionally, we also demonstrate both the practical applications and theoretical implications of the proposed dataset, including annotation agreement among the annotators using different annotation methods, which show significant agreement among the annotators.
- Finally, we present a baseline model to detect the severity of contradictions within these review pairs.
Not all peers are significant: A Dataset Exhaustive vs Trivial Scientific Peer Reviews Leveraging Chain-of-Thought Reasoning
Published at: Scientometrics, Springer(2025)
- We propose a novel dataset InsightfulPeer designed to classify peer reviews as either Exhaustive or Trivial, aimed at assessing the depth and quality of reviewer feedback.
- We implement multiple LLM variants (Llama-3.1, GPT-4, Mixtral-8x7b, and Gemma2-9b) to perform the classification task using CoT reasoning techniques.
- We conduct both qualitative and quantitative analyses to evaluate the fairness and effectiveness of these LLM variants in executing the task.
ConsistentPeer: Reviewers Through GraphRAG-Driven Counterfactuals to Measure Consistency in Peer Review
Published at: Scientometrics, Springer(2025)
- In this paper we proposed a novel pipeline to leverage graphs to visualize the relationships between review text, confidence score, rating and aspect categories
- Additionally, we also demonstrate both the practical applications and theoretical implications of the proposed pipeline, including the use of counterfactual reasoning to make informed decisions
- Finally, we present a complete pipeline to identify and resolve review text and it's cohesiveness with self annotated confidence score and rating.
Co-Reviewer: Are LLMs on the Same Page as Human Reviewers? An Agentic AI Framework for Evaluating Review Quality and Consensus
Published at: Scientometrics, Springer(2025)
- Developed Co-Reviewer, an agentic AI framework of four collaborative LLM agents designed to generate, evaluate, critique, and refine academic peer reviews.
- Conducted multi-dimensional evaluations comparing LLM-generated and human reviews across informativeness, sentiment, score consistency, and alignment with editorial decisions.
- Identified key LLM limitations and proposed improvements including domain-adaptive fine-tuning, structured critique generation, and hybrid human-AI review workflows.
LEDGE : Leveraging Dependency Graphs for Enhanced Context Aware Documentation Generation
Published at: Empirical Software Engineering, Springer(2025)
- We propose a novel approach to software documentation by leveraging GraphRAG, which integrates large language models (LLMs) with dependency graphs to generate structured, context aware documentation.
- In addition, we demonstrate both the practical applications and theoretical implications of the proposed approach, including its ability to improve software maintainability, improve knowledge transfer, and reduce the effort required for manual documentation.
- Finally, we present a comprehensive evaluation of our method on real world software projects, showcasing its effectiveness in generating more accurate, structured, and informative documentation compared to traditional approaches.
by Mihir Panchal