Human communication takes place in both spoken and written modes, as in face-to-face conversations, books reading, television watching and Internet browsing. However, for people with language disabilities (e.g. aphasia), people with low literacy, and people with poor command of a language, receiving and expressing information via a language is difficult. Because of the inability to comprehend words and/or to find words that express intended concepts, people with language barriers may encounter great challenges performing everyday tasks such as ordering food and visiting a doctor. As an alternative to words, pictures and sounds have been designed, tested, and used to evoke concepts in computer-based interfaces, education, industry, and advertisement. However, icons created by artists, user-uploaded photos, auditory icons and earcons cannot always satisfy the need for communicating everyday concepts for people with language barriers. Furthermore, the iconic vocabularies in current communication support systems have ambiguous representations and difficult to use vocabulary organization. My thesis research addresses issues in the above paragraph and explores building multimedia support for information delivery. The work is unique in that it provides different channels (visual and auditory) to support nonverbal communication, and that it has a disambiguated sense-level network structure with semantic links between related concepts. The development of such a multimedia-enhanced lexical database is challenging, due to the lack of guidance in the creation and evaluation of multimodal representations. To construct the lexical database, I went through a cycle of design, construction, evaluation, and modification for each media modality-to-concept association. I have introduced novel representations that are under investigated (e.g. videos and environmental sounds). I have explored difference sources of multimedia data (designed by specialists, designed by general users, and web resources not tailored for communication purposes). I have experimented with new methodologies for generating and evaluating multimedia-concept associations. I have also come up with essential guidelines for better multimedia vocabulary design. A communication support system, the Online Multimedia Language Assistant (OMLA), which employs various multimedia forms, including web images, icons, animations, videos, and environmental sounds, was built on top of my research on multimedia representations with the goal of assisting comprehension of common concepts across language barriers. OMLA is implemented as a popup dictionary in the form of a web browser extension. Users can select an unfamiliar word on a webpage to view its associated visual/auditory representation in a popup box. The Language Assistant can enhance concept understanding as people browse information on the Internet, and support face-to-face communication when people want to illustrate a term via a picture, a video or a sound. Through a series of studies, I verified that web images are as effective as stylized icons in conveying nouns, that videos outperformed other stimuli in illustrating verbs, and that nonspeech sounds are better in distinguishing many concepts like thunder, alarms, and sneezing. The Multimedia Augmented Online Language Assistant was shown to enhance information comprehension, and I am exploring its application in real life scenarios such as medical care.