Most standard Information Retrieval (IR) models primarily rely on keyword matching, which can be inadequate when a deeper contextual understanding is required. In such cases, it becomes essential to capture both lexical and semantic relationships between query-document pairs. To address this limitation, our team CodeWeavers proposes LexiSemIR, a two-stage re-ranking-based model developed for the CMIR-2025 (Code-Mixed Information Retrieval) shared task on Bengali-English code-mixed text. In the first stage, the top k documents are retrieved using a lexical bag-of-words model (BM25). These are then re-ranked in the second stage using a zero-shot bi-encoder, which computes semantic similarity between query and document embeddings. The proposed approach balances simplicity and performance, while minimizing trainable parameters due to its zero-shot design. LexiSemIR secured 3rd place in the CMIR-2025 shared task, achieving MAP = 0.1546 and P@5 = 0.38, thereby outperforming the BM25 baseline in early precision. The results highlight the model’s ability to effectively combine lexical and semantic retrieval strategies for robust performance in code-mixed IR settings.