Impairment in social communication skills is one of the core deficits in the children with Autism Spectrum Disorder (ASD). These children are characterized by an inherent inability to express their affective states thereby imposing limitations on traditional self-report and observational methodologies. However physiological signals are continuously available and are arguably not impacted by these difficulties. In recent years several assistive technologies utilizing the benefits of physiology-based systems have been investigated to promote social communication skills in this population. Among these we chose Virtual Reality (VR) as our platform. Investigations in the area of Human Computer Interaction (HCI) have shown that variations in the physiological signals can be evoked by different amounts of presence in the VR environment and the transition from one affective state to another is accompanied by dynamic shift in indicators of Autonomic Nervous System (ANS) activity. The presented work seeks to fuse behavioral viewing pattern and peripheral physiological features with the affective rating. Thus, this is a step towards indicating the potential of such a system to build an intelligent therapist-like affect-recognizer. The preliminary findings of a usability study are promising.