Artificial Intelligence (AI) is rapidly evolving technology and has proven its potential to transform the different sectors for societal good. However, this innovation also proffers ethical, legal, and social challenges that need to be addressed prudently. Therefore, it is essential to ensure that AI development and deployment is aligned with the best practices and principles of Responsible AI focusing on equality, accountability, transparency, privacy, safety, and respect towards human values. To achieve this aspiration, there is a need for a comprehensive and robust “AI Impact Assessment Mechanism” that can monitor and evaluate the AI systems and applications in India. Such a mechanism would enable the government, to have a clear and holistic view of the AI landscape in India, and to ensure that the AI systems are deployed with the existing legal framework and ethical norms governing AI in country. In alignment with the provisions of National Strategy on AI 2018 and responsible AI Principles adopted by the Indian Government and International Organizations like OECD and UNESCO, the study tends to develop a model of “AI Impact Assessment Mechanism” that will ultimately promote and facilitate effective governance of AI technology in the country.