A key topic in the field of computer vision is image classification, which involves predicting one class for each input image. Additionally, one of its tasks is the categorization of materials from images, which is difficult for both human and computer systems since materials might appear differently based on their surface characteristics, lighting geometry, viewing geometry, camera settings, etc. The revolutionary image classification architecture, deep convolutional neural networks (CNN) has shown promising results as compared to hand-crafted computer vision methods in recent studies for material classification. However, the number of material datasets that mimic the behavior of the real-world material is limited. To this end, our two contributions are reported. We proposed a new material dataset where images were acquired with larger acquisition settings. The dataset is developed in such a way that convolutional neural networks used to train on this dataset can produce features that can be adjusted with the varying appearance changes found in real-world material images. In order to integrate key features extracted from multiple perspectives of a same material sample, we proposed a distinct architecture that takes advantage of the current developments in multi-view learning techniques. We show that the proposed multi-view network can be used for both feature extraction and classification while significantly outperforming the traditional single-view network for material classification.