The aim of this study was to assess the effect of oil type on the network formation of heat-set protein aggregates in liquid oil. The gelling properties of such aggregates to structure oil into so-called ‘oleogels’ are related to both the particle–particle and particle–solvent interactions. To change these interactions, four different oils (medium chain triglyceride oil, sunflower oil, extra virgin olive oil and castor oil) differing in polarity were used. The rheological properties of the protein aggregate oleogels were determined and compared to gels prepared with hydrophilic and hydrophobic colloidal silica particles at the same concentration. The results show that gel strength of the network formed by protein aggregates is affected by the polarity of the oil, resulting in weaker gels in more polar oils as a result of larger particle–solvent interactions. Similar results were obtained for hydrophilic silica particles. In the case of castor oil, the increased particle–solvent interactions though hydrogen bonds limited gel formation for all particle types. Large deformation rheology shows that protein oleogels exhibit a yielding behaviour under large deformation, but regenerate its elasticity quickly after deformation is reduced. The rapid recovery of the protein network and the fact that the interactions between the protein aggregates can be tuned by changing the characteristics of the oil, may be interesting features for various applications in foods.