When I include this header file in the parser file, I get redefinition errors like this:
parser.tab.h:64:5: error: redefinition of enumerator 'T_LPAREN'
Yes, because the bison-generated header contains its own enum
of the token-type symbols. The enumerators of that enum
collide with those of your enum
. One of the purposes is exactly to make the C compiler notice and noisily reject name collisions such as the ones you introduced. This is Bison working as intended.
I want to use these token definitions in other places without relying on the ones generated by Bison.
I don't see why. The token type symbols are part of the API of the generated parser. Bison puts them in the generated header to facilitate other components (usually just the lexer) interoperating with the generated parser. There should not be any issue with such components including the generated header (though strictly speaking, they are not required to do so), and the token type codes should not be of any interest to any other components.
Is it possible to make Bison utilize external token definitions?
If for some reason you want to manually control the token codes then you can do so in the (bison) declarations of the tokens. For example,
%token T_STAR 342
I don't see much point, however.
Other than that, no, to the best of my knowledge and doc-reading ability, Bison does not provide a means to defer to "external" token definitions. The tokens to be used in a given grammar must be declared, using the appropriate Bison syntax, in the grammar file.
More generally, Bison does not read or interpret C syntax any more than it needs to do to perform its job -- mainly to recognize the boundaries of C snippets embedded within in places where that is allowed by the syntax of Bison's own language. This language is not C, and it is not preprocessed via the C preprocessor.
How to avoid redefinition issues in Bison?
Rely on the token definitions Bison emits. Or if you insist on defining your own, then don't use those definitions together with Bison's, though this way is bound to cause you unnecessary trouble.
Is it possible to separate the lexer and parser like this?
No, not really. A Bison-generated parser must be paired with a lexer that is specific to it. In actual practice, these are designed and built together. The Bison language is already the abstraction layer for the parser side of that. Although in principle, it would have been possible to give Bison the ability to accept external token declarations, that doesn't make much sense, because it is the grammar file that is authoritative for what token definitions are needed.
You could use some (even higher level) code generator to generate your grammar file with the wanted token type codes, but I really don't see what you stand to gain from any of this. Just declare the token types in the Bison grammar file, and use its generated header in any C sources that want to reference them.