From Simulation to Reality: CNN Transfer Learning for Scene Classification

Research output: Chapter in Book/Published conference outputConference publication

Abstract

In this work, we show that both fine-tune learning and cross-domain sim-to-real transfer learning from virtual to real-world environments improve the starting and final scene classification abilities of a computer vision model. A 6-class computer vision problem of scene classification is presented from both videogame environments and photographs of the real world, where both datasets have the same classes. 12 networks are trained from 2, 4, 8, , 4096 hidden interpretation neurons following a fine-tuned VGG16 Convolutional Neural Network for a dataset of virtual data gathered from the Unity game engine and for a photographic dataset gathered from an online image search engine. 12 Transfer Learning networks are then benchmarked using the trained networks on virtual data as a starting weight distribution for a neural network to classify the real-world dataset. Results show that all of the transfer networks have a higher starting accuracy pre-training, with the best showing an improvement of +48.34% image classification ability and an average increase of +38.33% for the starting abilities of all hyperparameter sets benchmarked. Of the 12 experiments, nine transfer experiments showed an improvement over non-transfer learning, two showed a slightly lower ability, and one did not change. The best accuracy overall was obtained by a transfer learning model with a layer of 64 interpretation neurons scoring 89.16% compared to the non-transfer counterpart of 88.27%. An average increase of +7.15% was observed over all experiments. The main finding is that not only can a higher final classification accuracy be achieved, but strong classification abilities prior to any training whatsoever are also encountered when transferring knowledge from simulation to real-world data, proving useful domain knowledge transfer between the datasets

Original languageEnglish
Title of host publication2020 IEEE 10th International Conference on Intelligent Systems, IS 2020 - Proceedings
EditorsVassil Sgurev, Vladimir Jotsov, Rudolf Kruse, Mincho Hadjiski
PublisherIEEE
Pages619-625
Number of pages7
ISBN (Electronic)9781728154565
DOIs
Publication statusPublished - 18 Sep 2020
Event10th IEEE International Conference on Intelligent Systems, IS 2020 - Sofia, Bulgaria
Duration: 28 Aug 202030 Aug 2020

Publication series

Name2020 IEEE 10th International Conference on Intelligent Systems, IS 2020 - Proceedings

Conference

Conference10th IEEE International Conference on Intelligent Systems, IS 2020
CountryBulgaria
CitySofia
Period28/08/2030/08/20

Keywords

  • Autonomous Perception
  • Computer Vision
  • Deep Learning
  • Environment Recognition
  • Scene Classification
  • Sim-to-real
  • Transfer Learning

Fingerprint

Dive into the research topics of 'From Simulation to Reality: CNN Transfer Learning for Scene Classification'. Together they form a unique fingerprint.

Cite this