In September 1331, Charles made an alliance with Otto the Merry, Duke of Austria, against Bohemia. He also sent reinforcements to Poland to fight against the Teutonic Knights and the Bohemians. In 1332 he signed a peace treaty with John of Bohemia and mediated a truce between Bohemia and Poland. In summer 1335, the delegates of John of Bohemia and the new King of Poland, Casimir III, entered into negotiations in Trencsén to put an end to the conflicts between the two countries. With Charles's mediation, a compromise was reached on 24 August: John of Bohemia renounced his claim to Poland and Casimir of Poland acknowledged John of Bohemia's suzerainty in Silesia. 

On 3 September, Charles signed an alliance with John of Bohemia in Visegrád, which was primarily formed against the Dukes of Austria. Upon Charles's invitation, John of Bohemia and Casimir of Poland met in Visegrád in November. During the Congress of Visegrád, the two rulers confirmed the compromise that their delegates had worked out in Trencsén. The three rulers agreed upon a mutual defence union against the Habsburgs, and a new commercial route was set up to enable merchants travelling between Hungary and the Holy Roman Empire to bypass Vienna.

The Babonići and the Kőszegis made an alliance with the Dukes of Austria in January 1336. John of Bohemia, who claimed Carinthia from the Habsburgs, invaded Austria in February. Casimir III of Poland came to Austria to assist him in late June. Charles soon joined them at Marchegg. The dukes sought reconciliation and signed a peace treaty with John of Bohemia in July. Charles signed a truce with them on 13 December, and launched a new expedition against Austria early the next year. He forced the Babonići and the Kőszegis to yield, and the latter were also compelled to hand over to him their fortresses along the frontier in exchange for faraway castles. Charles's peace treaty with Albert and Otto of Austria, which was signed on 11 September 1337, forbade both the dukes and Charles to give shelter to the other party's rebellious subjects.