In June 1960,[11] Baxter joined the Glasgow team Rangers for a transfer fee of £17,500, a Scottish record at the time.[7][12] He played for Rangers from 1960 to 1965, mainly as an attacking left half. During this period the team won the Scottish League Championship in 1961, 1963 and 1964,[12] and were winners of three Scottish Cups and four League Cups from as many finals, Baxter playing in all seven ties.[13] Rangers fans remember him as "Slim Jim", and in 18 "Old Firm" games against local rivals Celtic – ten Scottish League, five League Cup and three Scottish Cup matches – he was only twice on the losing side.[14]
His first Rangers game was in August 1960 at inside left against Partick Thistle in the Scottish League Cup. He scored his first goal for the club in November 1960, against Clyde,[7] and in the same month scored an early goal in Rangers' 8–0 win over the German team Borussia Mönchengladbach.[12] In 1961 Baxter played in the Rangers team that contested the first ever European Cup Winners' Cup Final, a two-legged tie that they lost 4–1 on aggregate to Fiorentina.[15]
In December 1964, Baxter played brilliantly to set up a 2–0 win for Rangers in an away game against Rapid Vienna in the European Cup. With the game won, rather than play out time Baxter's confidence over-extended into arrogance to make fun of his opponents with the ball. In the last minute he went to 'nutmeg' an opponent, who was having none of it and broke Baxter's leg with his challenge.[13] This was to be a watershed for Baxter. During the four months in which he was unable to play he began drinking to his detriment; this ultimately was to curtail his playing career and his life.[8]