Note that we do not need such a rule for identifiers because they will always be either terminated by a space, a punctuator, or an invalid character. Space and punctuator are both valid cases. The invalid character will be caught by another rule specifically for invalid characters. I will cover this case when I add tests for valid character set.
Review URL: https://codereview.appspot.com/5978048

git-svn-id: https://angleproject.googlecode.com/svn/trunk@1047 736b8ea6-26fd-11df-bfd4-992fa37f6226