To interpret sensor signals like images, image sequences, or continuous speech the representation and use of task-specific knowledge is necessary. The paper scetches a framework for the representation and utilization of declarative and procedural knowledge using a suitable definition of a semantic network. To meet the needs of machine-human interaction we extend this framework in two ways. A temporal model similar to Bruce is incorporated and representational structures are integrated to formulate qualitative (relational) knowledge. The problem-independent inference rules are extended to alllow for the temporal prediction and the dynamic refinement of this knowledge. Our integration of relational knowledge exemplarily shows how semantic network representations can benefit from developments in qualitative reasoning research.