My research is centered on harnessing the power of large pre-trained models or foundation models, such as GPT-3/4, to make them more interpretable, controllable, and data-efficient across a range of real-world applications. By infusing LLMs with knowledge-centric principles, I focus on the following key aspects to refine.