Processing and Analyzing Big data using Hadoop
Keywords:
Big data, remote sensing, DPU, HadoopAbstract
The benefits of remote access advanced the world day by day, create enormous volume of continuous information ( for the most part alluded to the expression "Huge Data"), where understanding data has a potential importance if gathered and totaled viably. In today's period, there is an incredible arrangement added to ongoing remote detecting Big Data than it appears at initially, and separating the helpful data in a proficient way drives a framework toward a noteworthy computational difficulties, for example, to examine, total, and store, where information are remotely gathered. Keeping in perspective the aforementioned components, there is a requirement for planning a framework engineering that invites both real-time, and in addition disconnected from the net information handling. Along these lines, in this paper, we propose constant Big Data expository design for remote detecting satellite application. The proposed design contains three primary units, for example, 1) remote detecting Big Data securing unit (RSDU); 2) information preparing unit (DPU); and 3) information investigation choice unit (DADU). To begin with, RSDU secures information from the satellite and sends this information to the Base Station, where beginning preparing happens. Second, DPU assumes a fundamental part in engineering for proficient handling of constant Big Data by giving filtration, load adjusting, and parallel preparing. Third, DADU is the upper layer unit of the proposed design, which is in charge of assemblage, stockpiling of the outcomes, and era of choice in light of the outcomes got from DPU. The proposed design has the capacity of partitioning, burden adjusting, and parallel handling of just valuable information. In this manner, it results in proficiently dissecting continuous remote detecting Big Data utilizing earth observatory framework. Moreover, the proposed design has the capacity of putting away approaching crude information to perform disconnected from the net investigation on to a great extent put away dumps, when required.
References
Real-Time Big Data Analytical Architecture for Remote Sensing Application Muhammad Mazhar Ullah Rathore, Anand Paul, Senior Member, IEEE, Awais Ahmad, Student Member, IEEE,Bo-Wei Chen, Member, IEEE, Bormin Huang, and Wen Ji, Member, IEEE
D. Agrawal, S. Das, and A. E. Abbadi, “Big Data and cloud computing: Current state and future opportunities,” in Proc. Int. Conf. Extending Database Technol. (EDBT), 2011, pp. 530–533.
J. Cohen, B. Dolan, M. Dunlap, J. M. Hellerstein, and C. Welton, “Mad skills: New analysis practices for Big Data,” PVLDB, vol. 2, no. 2, pp. 1481–1492, 2009.
J. Dean and S. Ghemawat, “Mapreduce: Simplified data processing on large clusters,” Commun. ACM, vol. 51, no. 1, pp. 107–113, 2008.
H. Herodotou et al., “Starfish: A self-tuning system for Big Data analytics,” in Proc. 5th Int. Conf. Innovative Data Syst. Res. (CIDR), 2011, pp. 261–272.
K. Michael and K. W. Miller, “Big Data: New opportunities and new challenges [guest editors’ introduction],” IEEE Computer., vol. 46, no. 6, pp. 22–24, Jun. 2013.
X. Li, F. Zhang, and Y. Wang, “Research on Big Data architecture, key technologies, and it’s measures,” in Proc. IEEE 11th Int. Conf. Dependable Auton. Secure Comput., 2013, pp. 1–4.
R. A. Dugane and A. B. Raut, “A survey on Big Data in real-time,” Int. J.Recent Innov. Trends Comput. Commun., vol. 2, no. 4, pp. 794–797, Apr.2014.
X. Yi, F. Liu, J. Liu, and H. Jin, “Building a network highway for BigData: Architecture and challenges,” IEEE Netw., vol. 28, no. 4, pp. 5–13,Jul./Aug. 2014.
E. Christophe, J. Michel, and J. Inglada, “Remote sensing processing:From multicore to GPU,” IEEE J. Sel. Topics Appl. Earth Observ. RemoteSens., vol. 4, no. 3, pp. 643–652, Aug. 2011.
Y.Wang et al., “Using a remote sensing driven model to analyze effect of land use on soil moisture in the Weihe River Basin, China,” IEEE J. Sel.Topics Appl. Earth Observ. Remote Sens., vol. 7, no. 9, pp. 38923902, Sep. 2014.
“C. Eaton, D. Deroos, T. Deutsch, G. Lapis, and P. C. Zikopoulos, Understanding Big Data: Analytics for Enterprise Class Hadoop and Streaming Data. New York, NY, USA: Mc Graw-Hill, 2012.
R. D. Schneider, Hadoop for Dummies Special Edition. Hoboken, NJ, USA: Wiley,2012
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors contributing to this journal agree to publish their articles under the Creative Commons Attribution 4.0 International License, allowing third parties to share their work (copy, distribute, transmit) and to adapt it, under the condition that the authors are given credit and that in the event of reuse or distribution, the terms of this license are made clear.
