Later writings from the first centuries of the Church continue to indicate that Rome, and the successors of Peter who preside there, hold special place.

In AD 110, Ignatius of Antioch, writing to the church in Rome, says to them: "You have never envied any one; you have taught others. Now I desire that those things may be confirmed [by your conduct], which in your instructions you enjoin [on others]" (brackets in original, source: https://www.newadvent.org/fathers/0107.htm).  This shows that Ignatius, living in Syria, recognized the primacy of Rome's teaching authority.

Eusebius, a bishop and historian writing in the 4th century, gives us an excerpt from a letter written by the bishop Dionysius of Corinth to the Romans, indicating the role as shepherd and father the bishops of Rome had assumed over the whole church: "You Romans keep up the hereditary customs of the Romans, which your blessed bishop Soter has not only maintained, but also added to, furnishing an abundance of supplies to the saints, and encouraging the brethren from abroad with blessed words, as a loving father his children."

Dionysius likewise indicates a reverence for the writing received from Pope Soter and from the previous bishops of Rome, particularly Clement: "Today we have passed the Lord's holy day, in which we have read your epistle. From it, whenever we read it, we shall always be able to draw advice, as also from the former epistle, which was written to us through Clement" (source: https://www.newadvent.org/fathers/250104.htm)

In the work of Irenaeus, Against Heresies, written in AD 189, rebukes heretics by appealing to the teaching authority of the church of Rome: "[we do this, I say,] by indicating that tradition derived from the apostles, of the very great, the very ancient, and universally known Church founded and organized at Rome by the two most glorious apostles, Peter and Paul; as also [by pointing out] the faith preached to men, which comes down to our time by means of the successions of the bishops. For it is a matter of necessity that every Church should agree with this Church, on account of its preeminent authority" (source: https://www.newadvent.org/fathers/0103303.htm, brackets in original).

Irenaeus also gives us an accounting of the succession of popes, from Peter down to his own day.

I could go on, as the examples are numerous, but the point is that, both in Scripture itself and in the earliest extant Christian writings outside of Scripture, the various churches around the world recognize that the office of chief steward, to which Peter was appointed, has been passed on to his successors in the See of Rome.