Top Research Papers on Data Engineering

Data-driven discovery is revolutionizing how we model, predict, and control complex systems. Now with Python and MATLAB®, this textbook trains mathematical scientists and engineers for the next generation of scientific discovery by offering a broad overview of the growing intersection of data-driven methods, machine learning, applied optimization, and classical fields of engineering mathematics and mathematical physics. With a focus on integrating dynamical systems modeling and control with modern methods in applied machine learning, this text includes methods that were chosen for their releva...

This work proposes several data engineering techniques to improve the performance of an analytical model while retaining the interpretability property, and illustrates the improvement in performance of these data engineering steps for popular analytical models on a real payment transactions data set.

An historical chronicle of how R became what it is today is provided, describing all its current features and capabilities, and the role of R in science in general as a driver for reproducibility is discussed.

A machine learning-guided paradigm that can use as few as 24 functionally assayed mutant sequences to build an accurate virtual fitness landscape and screen ten million sequences via in silico directed evolution is introduced.

A thorough review of approaches for automating data processing tasks in deep learning pipelines, including automated data preprocessing, as well as data augmentation (including synthetic data generation using generative AI methods and feature engineering), and the use of AutoML methods and tools to simultaneously optimize all stages of the machine learning pipeline are presented.

A framework for preparing Sentinel-1 SAR backscatter Analysis-Ready-Data in the Google Earth engine that combines existing and new Google Earth Engine implementations for additional border noise correction, speckle filtering and radiometric terrain normalization is presented.

A meta-analysis investigation of recent peer-reviewed GEE articles focusing on several features, including data, sensor type, study area, spatial resolution, application, strategy, and analytical methods confirmed that GEE has and continues to make substantive progress on global challenges involving process of geo-big data.

This study supplies contributions to the existing literature with a state-of-the-art bibliometric review of sustainable industrial and operation engineering as the field moves toward Industry 4.0, and guidance for future studies and practical achievements. Although industrial and operation engineering is being promoted forward to sustainability, the systematization of the knowledge that forms firms’ manufacturing and operations and encompasses their wide concepts and abundant complementary elements is still absent. This study aims to analyze contemporary sustainable industrial and operations e...

This work proposes the ChatExtract method, a method that can fully automate very accurate data extraction with minimal initial effort and background, using an advanced conversational LLM, and shows that approaches similar to ChatExtract are likely to become powerful tools for data extraction in the near future.

This study aims to comprehensively explore different aspects of the GEE platform, including its datasets, functions, advantages/limitations, and various applications, and observed that Landsat and Sentinel datasets were extensively utilized by GEE users.

An algorithm is presented that exploits all available Sentinel-1 SAR images in combination with historical Landsat and other auxiliary data sources hosted on the GEE to rapidly map surface inundation during flood events, relying on multi-temporal SAR statistics to identify unexpected floods in near real-time.

A lane change prediction framework for feature learning, with the aim to have a deep and comprehensive understanding of lane change behaviors, and reach a high performance based on the selected features.

By adding physics-based models to machine learning, it is possible not only to improve the performance of the purely black-box machine learning models, but also to make them more transparent and interpretable.

Remaining useful life (RUL) estimation has been intensively studied, given its important role in prognostics and health management (PHM) of industry. Recently, data-driven structures such as convolutional neural networks (CNNs), have achieved outstanding RUL prediction performance. However, conventional CNNs do not include an adequate mechanism for adaptively weighing input features. In this paper, we propose a double attention-based data-driven framework for aircraft engine RUL prognostics. Specifically, a channel attention-based CNN was utilized to apply greater weights to more significant f...

As our society becomes increasingly dependent on engineering and technology, it is more important than ever that everyone have a basic understanding of what engineers do, and the uses and implications of the technologies they create. Yet few citizens are technologically literate, in large part because technology and engineering are not taught in our schools 1 .

The increasing availability of condition-monitoring data for components/systems has incentivized the development of data-driven Remaining Useful Life (RUL) prognostics in the past years. However, most studies focus on point RUL prognostics, with limited insights into the uncertainty associated with these estimates. This limits the applicability of such RUL prognostics to maintenance planning, which is per definition a stochastic problem. In this paper, we therefore develop probabilistic RUL prognostics using Convolutional Neural Networks. These prognostics are further integrated into maintenan...

Non-volatile memory devices, which offer large capacity and mechanical dependability as a mainstream technology, have played a key role in fostering innovation in modern electronics. Despite the advantages of non-volatile memory devices, their low ON/OFF ratio and slow operational speed have limited their performance compared to their volatile counterparts. In this study, we present a non-volatile floating-gate memory device based on van der Waals heterostructures , which exhibits ultrahigh-speed memory operations in the range of a hundred nanoseconds with an extinction ratio of up to 10 6 . T...

Engineering is a difficult profession to explain to the average person, much less student, and is probably one of the most frequently misunderstood. The session described in this paper was developed to put engineering in common terms for the lay person, as well as provide an interesting and fun way to explore different concentration areas of the profession. The demonstration has been given to children as young as six years old, to parents of incoming engineering freshman and to emeritus engineers for the purpose of highlighting how the profession has changed. Little adaptation is needed, surpr...

These results were fundamental in indicating areas with higher rates of change in a long time series in the Brazilian Cerrado and to highlight the challenges of mapping distinct NV types in a highly seasonal and heterogeneous savanna biome.

Designed to cover the fundamental concepts of thermodynamics used in engineering, the book introduces topics such as the laws of thermodynamics, exergy analysis, thermodynamic cycles, measurement theory, and applications. Using step by step examples and numerous illustrations, the book is designed with a self-teaching methodology, including a variety of exercises with corresponding answers to enhance mastery of the content. The book provides an engineer with a basic understanding or review of thermodynamic principles. Features: Designed to cover the fundamental concepts of thermodynamics used ...