This is what Camstudio looks like when button view is activated. To return to the normal window, simply click the double letter icon next to the stop button.

If you change your view options to buttons view, there is a good chance your Camstudio window will dissapear(or fail to expand) if you minimize the window. In order to end the current session and restart the application:

 Go to Start 

Type in task manager

Click view running processes with task manager

Find Recorder.exe, then click the end process button from the bottom right of the window

Open up Camstudio to start fresh