The symbol gets inserted whenever you hold down Shift key while pressing the Space key. Shift + Space is the shortcut key for the symbol. If you need to be able to hold down Shift + Space keys while inserting regular space symbol, you will need to disable this shortcut.
To disable or change this shortcut, on the Options menu, click Keyboard Shortcuts and then locate and change or remove the Shift + Space shortcut.