Self-awareness is not likely to be a feasible or useful attribute of a program that just computes an answer. It is more likely to be feasible and useful for programs that maintain a persistent activity.
What kind of program would be analogous to Pat deciding while on the way to his job that he needed cigarettes? See formula (4) below. Here are some possibiliities.
It's not clear what event in a computer program might correspond to Pat's sudden need for cigarettes. The following examples don't quite make it.
A specialized theorem-proving program is being operated as a subprogram by a reasoning program . Assume that the writer of has only limited knowledge of the details of , because is someone else's program. might usefully monitor the operation of and look at the collection of intermediate results has produced. If too many of these are redundant, may restart with different initial conditions and with a restriction that prevents sentences of a a certain form from being generated.
An operating system keeps track of the resources a user is using, and check for attempts to use forbidden resources. In particular it might check for generation of password candidates. In its present form this example may bad, because we can imagine the checking be done by the programs that implement supervisor calls rather than by an inspector operating with clock interrupts. While the programs called by clock interrupts exhibit a simple form of self-awareness, the applications I know about are all trivial.
The main technical requirement for self-awareness of ongoing processes in computers is an interrupt system, especially a system that allows clock interrupts. Hardware supporting interrupts is standard on all computers today but didn't become standard until the middle 1960s.6 The human brain is not a computer that executes instructions in sequence and therefore doesn't need an interrupt system that can make it take an instruction out of sequence. However, interruption of some kind is clearly a feature of the brain.
With humans the boundary between self and non-self is pretty clear. It's the skin. With computer based systems, the boundary may be somewhat arbitrary, and this makes distinguishing self-awareness from other awareness arbitrary. I suppose satisfactory distinctions will become clearer with experience.