Penal substitution is the idea that sin we broke God's law, His justice demanded that we be punished, and Christ satisfied God's justice by a substitutionary penalty. Sometimes this idea is fairly explicit in the Fathers. In particular, I found Eusebius' statement that in OT sacrifices, animals were slain in the place of men, prefiguring what Christ would do, as a very powerful affirmation of penal substitution. Clearly, the OT sacrifices were offered to God, not Satan, so if you view the sacrifices as penal substitutions, you are basically there. The other person that stood out to me was Theodore Abucara, who plainly taught penal substitution. While he was pre-reformation, he is probably too late to be considered a church father.
Beyond Eusebius and Abucara, the idea of penal substitution is in the Fathers, but it's in pieces; one father talking about Christ paying a penalty, another saying He satisfies the Father, another saying He took our place or was substituted. And that's perhaps why I often hear the Reformers took the best elements of the Fathers and assembled them in a systematic way. After all, the fathers didn’t write that many systematic theologies.
One final lesson I learned from the study; don't throw out a father because he taught the ransom theory or at times describes Christ's work as in medical terms rather than judicial terms. Abucara taught me that the ransom theory (Christ's rescuing us from Satan's power) is compatible with penal substitution. There are just two different views of the atonement - but they are not fundamentally at odds with each other. Similarly, the view that Christ healed our sickness due to sin is not mutually exclusive with the idea that He was a penal substitute.
But enough from me, let's hear from the church fathers!
Turrtullian (160-220): But He has both suffered the penalty in our presence, and surrendered His life, laying it down for our sakes, and is held in contempt by the Gentiles
Against Marcion, Book IV, Chapter 14
Eusebius of Caesarea (c. 265-c. 340)
But he alone having reached our deep corruption, he alone having taken upon himself our labors, he alone having suffered the punishments due for our impieties, having recovered us who were not half dead merely, but were already in tombs and sepulchers, and altogether foul and offensive, saves us, both anciently and now, by his beneficent zeal, beyond the expectation of any one, even of ourselves, and imparts liberally of the Father's benefits—he who is the giver of life and light, our great Physician and King and Lord, the Christ of God. Church History. Book X.
And how can He make our sins His own, and be said to bear our iniquities, except by our being regarded as His body, according to the apostle, who says: "Now ye are the body of Christ, and severally members?" And by the rule that "if one member suffer all the members suffer with it," so when the many members suffer and sin, He too by the laws of (c) sympathy (since the Word of God was pleased to take the form of a slave and to be knit into the common tabernacle of us all) takes into Himself the labours of the suffering members, and makes our sicknesses His, and suffers all our woes and labours by the laws of love. And the Lamb of God not only did this, but was chastised on our behalf, (d) and suffered a penalty He did not owe, but which we owed because of the multitude of our sins; and so He became the cause of the forgiveness of our sins, because He received death for us, and transferred to Himself the scourging, the insults, and the dishonour, which were due to us, and drew down on Himself the apportioned curse, being made a curse for us. And what is that but the price of our souls? And so the oracle says in our person: "By his stripes we were healed," and "The Lord delivered him for our sins," with the result that uniting Himself to us and us to Himself, and appropriating our sufferings, He can say, "I said, Lord, have mercy on me, heal my soul, (468) for I have sinned against thee," Demonstratio Evangelica. Book X.
And this thought, I hold, was not due to accident, nor was its source in man, but it was divinely suggested. For when they saw since they were holy, brought nigh to God, and enlightened by the Divine Spirit in their souls that there was need of great stress on the cleansing of the sons of men, they thought that a ransom was due to the source of life and soul in return for their own salvation. And then as they had nothing better or more valuable than their own life to sacrifice, in place of it they brought a sacrifice through that of the unreasoning beasts, providing a life instead of their own life. They did not consider this was sinful or unrighteous. They had not been taught that the soul of the brutes was like man's, which has discourse of reason: they had only learned that it was the animal's blood, and that in the blood is the principle of life, which they offered themselves, sacrificing as it were to God one life instead of another.
Moses makes this abundantly clear, when he says:
"For the life of all flesh is the blood, and I have given it to you upon the altar to make atonement for your sins: for the blood shall make atonement for the soul. Therefore I said to the children of Israel, No soul of you shall eat blood."
Note carefully in the above the words, "I gave to you upon the altar to make atonement for your souls; for the blood shall make atonement for the soul."
He says clearly that the blood of the victims slain is a propitiation in the place of human life. And the law about sacrifices suggests that it should be so regarded, if it is carefully considered. For it requires him who is sacrificing always to lay his hands on the head of the victim, and to bear the animal to the priest held by its head, as one offering a sacrifice on behalf of himself. Thus he says in each case:
"He shall bring it before the Lord. And he shall lay his hands on the head of the gift."
Such is the ritual in every case, no sacrifice is ever brought up otherwise….
And any Jews, of course, who have taken refuge in Christ, even if they attend no longer to the ordinances of Moses, but live according to the new covenant, are free from the curse ordained by Moses, for the Lamb of God has surely not only taken on Himself the sin of the world, but also the curse involved in the breach of the commandments of Moses as well. The Lamb of God is made thus both sin and curse—sin for the sinners in the world, and curse for those remaining in all the things written in Moses' law. And so the Apostle says: "Christ has redeemed us from the curse of the law, being made a curse for us"; and "Him that knew no sin, for our sakes he made sin."For what is there that the Offering for the whole world could not effect, the Life given for the life of sinners, Who was led as a lamb to the slaughter, and as a lamb to the sacrifice, and all this for us and on our behalf? …
He then that was alone of those who ever existed, the Word of God, before all worlds, and High Priest of every creature that has mind and reason, separated One of like passions with us, as a sheep or lamb from the human flock, branded on Him all our sins, and fastened on Hirn as well the curse that was adjudged by Moses' law, as Moses foretells: "Cursed is every one that hangeth on a tree." This He suffered "being made a curse for us; and making himself sin for our sakes."And then "He made him sin for our sakes who knew no sin,"and laid on Him all the punishments due to us for our sins, bonds, insults, contumelies, scourging, and shameful blows, and the crowning trophy of the Cross. …
Demonstratio Evangelica. Book I .
Macarius of Jerusalem (Bishop of Jerusalem from 312 to shortly before 335): But he himself came as the Saviour of all, and in our name bore, in his own flesh, the punishment owed by us. (Acts of the Council of Nice. Book 2)
And thus taking from our bodies one of like nature, because all were under penalty of the corruption of death He gave it over to death in the stead of all, and offered it to the Father— doing this, moreover, of His loving-kindness, to the end that, firstly, all being held to have died in Him, the law involving the ruin of men might be undone (inasmuch as its power was fully spent in the Lord's body, and had no longer holding-ground against men, his peers)
But since it was necessary also that the debt owing from all should be paid again: for, as I have already said , it was owing that all should die, for which special cause, indeed, He came among us: to this intent, after the proofs of His Godhead from His works, He next offered up His sacrifice also on behalf of all, yielding His Temple to death in the stead of all, in order firstly to make men quit and free of their old trespass, and further to show Himself more powerful even than death, displaying His own body incorruptible, as first-fruits of the resurrection of all. On the Incarnation of the Word.
And so it was that two marvels came to pass at once, that the death of all was accomplished in the Lord's body, and that death and corruption were wholly done away by reason of the Word that was united with it. For there was need of death, and death must needs be suffered on behalf of all, that the debt owing from all might be paid. 6. Whence, as I said before, the Word, since it was not possible for Him to die, as He was immortal, took to Himself a body such as could die, that He might offer it as His own in the stead of all, and as suffering, through His union with it, on behalf of all, “Bring to nought Him that had the power of death, that is the devil; and might deliver them who through fear of death were all their lifetime subject to bondage.” On the Incarnation of the Word.
For if He came Himself to bear the curse laid upon us, how else could He have
become Galatians 3:13 a curse,unless He received the death set for a curse? And that is the Cross. For this is exactly what is written: Cursed Deuteronomy 21:23 is he that hangs on a tree. On the Incarnation of the Word
Gregory Thaumaturgus (213-270): and in the glorious advent of the Son of God, who of the Virgin Mary took flesh, and endured sufferings and death in our stead, and came to resurrection on the third day, and was taken up to heaven; and in His glorious appearing yet to come; and in one holy Church, the forgiveness of sins, the resurrection of the flesh, and life eternal. (A Sectional Confession of Faith, Section 17)
Victor of Antioch (303) He was made like us; he took upon himself our miseries and crosses that he might raise up our nature, fallen through sin, and finally restore it to its former grade of dignity. The advantages, therefore, which have flowed to us thorugh his sufferings are very many; for he himself paid our debts for us, himself bore our sins, himself for our sake both suffered and groaned. (On Mark xv Cited in J.A. Cramer, Catenae in evangelia Matthaei et Marci (Oxford. 1840))
And so by his own blood he redemed those whom their own sins had sold. But Christ, sold by undertaking a condition, is not held by the price of a fault and sin, because he committed no sin. He therefore contracted the debt at our price, not by his own expenditure; he took away the handwriting, removed the userer, freed the debtor, alone paid that wihc was owed by all. (On the Patriarch Joseph, Ambrose. "De Joseph" PL 14.673–704 Chap. iv)
God so took flesh as to abolish the curse of sinful flesh, and was made a curse for us that blessing might absorb the curse, perfection the sin, pardon the sentance, life death. For he accepted death that the sentance might be fulfilled, and perfect satisfaction even unto death be made for him condemned through the curse of the flesh. Therefore nothing was done contrary to the sentence of God, since the condition of the divine sentence was fulfilled. For the curse extends even unto death, but after death is grace. (Concerning Esau, or concerning the Flight of the Age, Chap. vii Ambrose. Seven Exegetical Works. CUA Press, 1972 p. 314)
But how do the Pelagians say
that only death passed upon us by Adam's means? For if we die because he died, but he died because be sinned, they say that the punishment passed without the guilt, and that innocent infants are punished with an unjust penalty by deriving death without the deserts of death. This, the catholic faith has known of the one and only mediator between God and man, the man Christ Jesus, who condescended to undergo death— that is, the penalty of sin— without sin, for us. As He alone became the Son of man, in order that we might become through Him sons of God, so He alone, on our behalf, undertook punishment without ill deservings, that we through Him might obtain grace without good deservings. Because as to us nothing good was due so to Him nothing bad was due. Against two epistles of the Pelagians. Bk. iv.
The apostle boldly says of Christ, "He was made a curse for us;" for he could also venture to say, "He died for all." "He died," and "He was cursed," are the same. Death is the effect of the curse; and all sin is cursed, whether it means the action which merits punishment, or the punishment which follows. Christ, though guiltless, took our punishment, that He might cancel our guilt, and do away with our punishment. Against Faustus the Manicheaean, Bk. xiv
It is your fault that you are unjust; it is your punishment that you are mortal. That he might be your neighbor he undertook your punishment. He did not receive your sin. Or if he received it, he received it to destroy it, not to do it.... By receiving the punishment, and not receiving the sin, he has destroyed both sin and punishment. (Sermon in Luke, Sermon CLXXI III. PL XXXVII)
Proclus of Constantinople (d 446): The nature of man owed much in consequence of sins, and was in perplexity over the debt. For through Adam all had been accused of sin; the devil held us slaves; he made boast of having purchased us, employing for a proof our much suffering body. The evil falsifier of the passions stood pressing the debt upon us, and demanding justice from us. There was therefore need of one of two things, — either that death, arising from the condemnation, should be laid upon all, since also all sinned; or that such a payment should be made in recompense as to satisfy every righteous demand. A man, therefore, could not save us; for he lay under the debt of sin. An angel could not redeem humanity, for he did not know how to provide such a ransom. It remained, therefore, that the sinless God should die in behalf of those who had sinned. For this was the only deliverance from the evil left. What then? He that brought all nature from nothingness into being, who was not perplexed to find a way of delivery, found out for them that were condemned a most sure life, and release most becoming to death, and is made man of a virgin in such a manner as he himself knows, — for reason is not able to interpret the wonder, — and dies in what he became, and paid the ransom in what he was; as Paul says: "In whom we have redemption through his blood, the forgiveness of sins." Oh great work! he purchased immortality for others, for he was himself immortal. (Homily on the Nativity of Christ Proclus. C. Moss. Le Museon. 1929)
Gregory the Great (540-604)
But we must consider how He is righteous and ordereth all things righteously, if He condemns Him that deserveth not to be punished. For our Mediator deserved not to be punished for Himself, because He never was guilty of any defilement of sin. But if He had not Himself undertaken a death not due to Him, He would never have freed us from one that was justly due to us. And so whereas ‘The Father is righteous,’ in punishing a righteous man, ‘He ordereth all things righteously,’ in that by these means He justifies all things, viz. that for the sake of sinners He condemns Him Who is without sin; that all the Elect [electa omnia] might rise up to the height of righteousness, in proportion as He Who is above all underwent the penalties of our unrighteousness. Morals Book III
Theodore Abucara (d 770) God, in his just judgment, demanded of us all things that were written in the law, when we were not competent to pay them; for that reason our Lord paid them for us, and assumed the curse and condemnation to which we were exposed, and further took it upon himself. What things we ought to have suffered, he himself bore. — The same, in the same place: Now tell me, what those five enemies are from which Christ has liberated us.
A. Death; the devil; the curse of the law, and its condem- a nation; sin; and hell.
B. As far as pertain to death, you have said that this was destroyed by the obedience of Christ; so also you have told how it liberated us from servitude to the devil. Now tell us how he redeemed us from the curse of the law, having been made a curse for us ? — After a little:
A. God in his just judgment demanded of us all those things that were written in the law, when we were not competent to pay them; for that reason Christ our Lord paid them for us, and assumed the curse and condemnation to which we were exposed, and further took it upon himself, and himself bore what things we ought to have suffered, having been scourged, spit upon, smitten, struck upon the ears, crucified, and put to death for us. (Bishop of the Carians. Discussion xv. Chap. v. (Dialogues (PG, XCVII, 1461-1609))
Theophylact of Ohrid (1050-1109) When he had spoken of the majesty of the Divine Word, then he discoursed of the care which he took for men through his flesh, which was of much greater importance than that he sustains all things. Moreover, he lays down here two things, both that he cleansed us from sins, and that he did this through himself. For by the cross and death which he sustained he purged us, not only- because he died for our sin, though he was himself free from all sin, and because he paid the penalty, which nevertheless he did not owe, for us, and freed that nature, which was condemned simply because of the sin and transgression of Adam, etc. — On Chap. ix.: Christ died for this purpose that he might cleanse us, and left to us in his testament pardon of our fault, and the use of our Father's goods, having been made a Mediator of our Father. For the Father was not willing to bestow upon us the inheritance, but was angry with us, as sons rejecting him, and estranged from him. Christ, so made Mediator, reconciled him to us. How? He himself bore for us that which we ought to have suffered (for we deserved to die), and made us worthy of his testament (On those words (Heb. i.) " When he had by himself purged our sins" Migne, CXXIII-CXXIV)
Does any one say, If they have not each the sin of Adam, how do you assert that none is saved without satisfaction for the sin of Adam? For how does a just God exact from them satisfaction for that which they have not? To which I say: God does not exact from any sinner more than he owes; but since none can pay as much as he owes, Christ alone paid more than is due for all that are saved. (On the Conception of the Virgin, and Original Sin, Chap, xxi. Why God Became Man, and the Virgin Conception and Original Sin. Magi Books, 1969))
Anselm. To remit sin in this manner is nothing else than not to punish; and since it is not right to cancel sin without compensation or punishment; if it be not punished, then is it passed by undischarged.
Boso. What you say is reasonable.
Anselm. It is not fitting for God to pass over anything in his kingdom undischarged.
Boso. If I wish to oppose this, I fear to sin.
Anselm. It is, therefore, not proper for God thus to pass over sin unpunished.
Boso. Thus it follows
Cur Deus Homo. Book I, Chapter XII
Bernard de Clairvaux (1090-1113) It was a man who owed, a man who paid. For if one, he says, died for all, then all died, viz. that the satisfaction of one may be imputed to all, just as he alone bore the sins of all; nor is there any one found to purchase, and another one to make satisfaction, because one Christ is head and body. The head, therefore, made satisfaction for the members, Christ for his own bowels. (Epistle cxc to Innocent:, Tractate on the Errors of Abelard. PL, CLXXXII, 1062b
Nicolaus of Cusa (1401-1464) For our justification did Christ so do. For we, sinners, in him paid the penalties of hell, which we justly deserved. (Cardinal, Excitationes, Book 10 (In Opera Omnia. Hamburg: Felix Meiner, 1983))