With the rising incorporation of technology in the legal sphere, whether it is research work, drafting,pleading, or even filing of cases, itis evidentthatsoftware usingArtificial Intelligence (hereinafter known as 'AI') will be widely used in future. Ten years down the line, we may enter an era where an AI Bot has replaced Judges in petty cases. Further, such technology may be deployed in public places or high-security areas that recognise a person's facial features to predict that they are about to commit a crime and, in turn, alert the authorities beforehand.AImay, on the one hand, come across as an efficient, impartial and cost-effective tool in the justice delivery system, but it may also pose several challenges for the lawmakers pertaining to culpability, ethics and threats of generative AI. Therefore, this paper aims to examine the utility & challenges arising from integrating AI-based technology in judicial administration. Apart from navigating the utility and risks of using such technology, the paper offers viable solutions for the said challenges. The methodology adopted for this paper is a mix of Doctrinal and Socio-Legal Research Approach.