Reusing building materials and concrete of old buildings can be a promising strategy for sustained development. In buildings, the performance of materials under elevated temperatures is of particular interest for determining fire resistance. In this study, the effect of pozzolan and aggregate type on properties of concrete exposed to fire was investigated. In doing so, nanosilica with cement-replacement levels of 0, 2, and 4% as well as silica fume and ultrafine fly ash with cement-replacement levels of 0, 7.5, and 15% were used to study effect of pozzolan type, and recycled refractory brick (RRB) fine aggregate replacing natural fine aggregate by 0 and 100% was utilized to explore effect of aggregate type. A total of 126 cubic concrete specimens were manufactured and then investigated in terms of compressive strength, ultrasonic pulse velocity, and weight loss at $23^{\circ}C$ and immediately after exposure to 400 and $800^{\circ}C$. Results show that replacing 100% of natural fine aggregate with recycled refectory brick fine aggregate in the concretes exposed to heat was desirable, in that it led to a mean compressive strength increase of above 25% at $800^{\circ}C$. In general, among the pozzolans used here, silica fume demonstrated the best performance in terms of retaining the compressive strength of heated concretes. The higher replacement level of silica fume and ultrafine fly ash pozzolans in the mixes containing RRB fine aggregate led to a greater weight loss rate, while the higher replacement level of nanosilica reduced the weight loss rate.