Two things are central from my point of view. First, heterodox economists are concerned with the amplified reproduction of society, and this implies that the production and distribution of the social surplus is central for their theories. This is part of a tradition that harks back to classical political economy. The determinantion of the surplus implies that distribution is determined exogenously by social and institutional conditions (be that the real wage as affected by the bargaining position of the labor class, or the rate of profit as determined by the monetary rate of interest influenced by the central bank). Further, the determination of surplus, for an externally determined distribution, and a given technology, provides an explanation of relative prices (value).
It is important to note that there are several particular theoretical ways of approaching each of these questions associated to the reproduction of the economy. For example, some Marxists emphasize that relative prices are proportional to the amounts of labor directly and indirectly needed to produce the commodities. Sraffa provides a different approach, that is compatible with Marx's views of the working of the economy. Post Keynesian groups that emphasize the determination of prices according to full cost pricing are also compatible with this general preocupation about the determination of the social surplus. In other words, several schools of thought are heterodox in the acceptance of the necessity of understanding how the surplus is generated and distributed, even if they have different theories (that are not always compatible among them).
The second essential proposition that defines heterodoxy is related to the theory of output and employment determination. Heterodox economists believe that output is demand determined. That is, autonomous spending determines the level of activity. There are a few implications to this simple proposition. First, the level of autonomous spending will only generate full employment of productive resources by chance, and unemployment is a permanent feature of the economic system. Second, as the level of income equilibrates savings to investment, the rate of interest must be a monetary (not real phenomenon). Additionally, heterodox economists argue that effective demand is valid in the long run.
In sum, if one believes that prices reflect, for a given technology, the way classes struggle for higher income shares within the process of reproducing the material conditions for survival (including processes in which there is accumulation), and one believes that output and the process of accumulation are driven by the exogenous forces of demand, one may be called heterodox.
Some other issues are often seen as central for defining heterodox economics. Endogenous money is a typical example. And it is true that most, if not all, heterodox economists follow some version of endogenous money theory. However, it is clear that even if money is not endogenous, the rate of interest is still a monetary variable in heterodox economics (e.g. in Keynes' General Theory). Also, several mainstream economists have endogenous money, from Wicksell to those using the Taylor rule now. The same could be said about true or non-probabilistic uncertainty. It is important, but neither the determination of prices of production or effective demand are directly affected by uncertainty. And several Austrians (hardly heterodox according to the criteria above) are very fond of the idea of uncertainty. The last additional issue that sometimes is used to define heterodoxy is complexity (or non-ergodicity), but is the least relevant. The main difference between heterodox and orthodox are related to causality issues, and they can be reproduced in simple or complex theoretical frameworks.
The obvious question that anyone may pose is why does it matter to define heterodox economics precisely. For example, Colander, Holt and Rosser (CHR) suggest that heterodox economists should not consider themselves heterodox but just economists, and try to influence the mainstream working with the best among the orthodox. The problem is that logic and evidence can support either prices of production or supply and demand, and, by the same token, either Say's Law or Effective Demand must be wrong. In economic policy compromise might be possible, but in theory principles cannot be compromised. I replied to their paper here.
PS: My paper is published in the Journal of Post Keynesian Economics, with CHR's reply (subscription required).