In this paper, the effect of the type and amount of fibers on the physicomechanical properties of concrete containing fine recycled refractory brick (RRB) and natural aggregate subjected to elevated temperatures was investigated. For this purpose, forta-ferro (FF), polypropylene (PP), and polyvinyl alcohol (PVA) fibers with the volume fractions of 0, 0.25, and 0.5%, as well as steel fibers with the volume fractions of 0, 0.75, and 1.5% were used in the concrete containing RRB fine aggregate replacing natural sand by 0 and 100%. In total, 162 concrete specimens from 18 different mix designs were prepared and tested in the temperature groups of 23, 400, and 800. After experiencing heat, the concrete properties including the compressive strength, ultrasonic pulse velocity (UPV), weight loss, and surface appearance were evaluated and compared with the corresponding results of the reference (unheated) specimens. The results show that using RRB fine aggregate replacing natural fine aggregate by 100% led to an increase in the concrete compressive strength in almost all the mixes, and only in the PVA-containing mixes a decrease in strength was observed. Furthermore, UPV values at for all the concrete mixes containing RRB fine aggregate were above those of the natural aggregate concrete specimens. Finally, regarding the compressive strength and UPV results, steel fibers demonstrated a better performance relative to other fiber types.