If you want to travel from Georgia to Azerbaijan by train, buy your train ticket locally at a train station in Georgia.
There is one daily overnight train from Tbilisi (Georgia) to Baku (Azerbaijan).
To travel from Georgia to Bulgaria, first travel by train to Poti (Georgia). In Poti you walk from the railway station to the port. Change to the ferry to Varna (Bulgaria). The journey time is about 54 hours. A ticket starts from Euro 100. In Varna take a taxi or bus to the railway station and continue your travels.
If you want to travel from Georgia to Turkey only by train, you have to make a detour via Russia. Therefore a bus connection from Georgia to Turkey is a much faster option.
Therefore, take first a train from Tbilisi (Georgia) to Batumi (Georgia).
One bus line is from Batumi (Georgia) to Trabzon (Turkey) with at least four connections daily for about EUR 15. From Batumi (Georgia) to Istanbul (Turkey) at least two connections daily for about EUR 25. Buy this bus ticket online at Metro Turizm.
In Trabzon (Turkey) you do not have a connection to the rail network of Turkey. Therefore, the easiest way is to continue with the bus to Ankara or Istanbul.
An other option is by bus from Batumi (Georgia) to Kars (Turkey). In Kars (Turkey) you change to the train to Ankara and Istanbul.
Border crossing is currently: you leave the bus with the luggage before the border, cross the border by foot (easy) and after the border the bus picks you up again.
In future there will be a train connection from Georgia to Turkey, but it is still under construction.
To travel from Georgia to Ukraine, first travel to Batumi (Georgia). Walk from the railway station to the port. From there take the ferry to Chornomorsk (Ukraine). The journey time is about 48 hours. Ticket price starts from USD 110. In Chornomorsk which is located 20 kilometres south of Odessa, take a taxi, mini-bus or bus from the port to the railway station of Odessa.