Big Data Case Studies
1
Introduction
2
Getting Started
2.1
Sensor Dev Env
2.1.1
Software Setup
2.1.2
Alternative Software Setup (building releases)
2.1.3
Hardware Setup
2.2
Gateway Dev Env
2.2.1
Raspberry connection via screen
2.2.2
Raspberry connection without a screen (headless setup)
2.2.3
Set up PuTTY and finalize connection to Raspberry
3
Setup a Jupyterhub on your RaspberryPi
3.1
Prepare Python environment
3.2
Install JupyterHub
3.3
Configure JupyterHub as a system service
4
PyCharm
4.1
PyCharm Git Repository
4.2
RemoteDeploy
5
Getting Started Package
5.1
Raspbery Pi OS “Buster”
5.2
Board
5.2.1
Raspberry Pi 4 - Model B
5.2.2
Compute Model 4
Documentation
6
Sensor
6.1
Initializing acceleration logging
6.2
Retrieving data from FIFO
6.3
Using the data by heartbeat
6.4
Initialization during boot
6.5
Streaming of acceleration data
6.6
Saving the config inside the flashdb
6.7
Heartbeat
6.8
Implementation
6.8.1
app_heartbeat.c
6.8.2
ruuvi_task_sensor.c
6.8.3
app_accelerometer_logging.c
6.8.4
app_comms.c
6.8.5
ruuvi_nrf5_sdk15_communication_ble_gatt.c
6.8.6
app_config.h
6.8.7
app_sensor.c
6.8.8
main.c
6.8.9
ruuvi_interface_lis2dh12.c
6.8.10
ruuvi_nrf5_sdk_rtc_mcu.c
6.8.11
ruuvi_nrf5_sdk15_power.c / ruuvi_nrf5_sdk15_power.h
6.8.12
ruuvi_task_flash_ringbuffer.c
6.8.13
ruuvi_task_flashdb.c
6.8.14
app_button.c
6.9
Energy consumption
6.10
Firmware packaging
7
External Flash Integration
7.1
Macronix Flash
7.2
Database vs. Filesystem
7.2.1
FlashDB
7.3
Implementation
7.3.1
FlashDB API
7.3.2
Data Base config files
7.3.3
Macronix low level driver functions
7.4
FlashDB evaluation
7.4.1
Amount of data wich can be stored
7.4.2
Energy consumption
7.4.3
GitHub Issue #11 Write Performance on External Flash
8
Gateway
8.1
Sensor Gateway Communication Library
8.1.1
Activate acceleration logging
8.1.2
Deactivate acceleration logging
8.1.3
Set sensor configuration
8.1.4
Set sensor time
8.1.5
Get sensor configuration
8.1.6
Get sensor time
8.1.7
Get acceleration data
8.1.8
Get flash statistics
8.1.9
Get logging status
8.2
Advertisement logging
8.3
Data storage format
8.3.1
Csv file acceleration Logging
8.3.2
Csv file advertisement Logging
9
Backend
10
BLE GATT Messages
10.1
General message structure
10.2
Control messages
10.2.1
Set heartbeat
10.2.2
Get heartbeat
10.2.3
Start transmitting acceleration data
10.2.4
Set configuration of acceleration sensor
10.2.5
Read configuration of acceleration sensor
10.2.6
Set system time
10.2.7
Read system time
10.2.8
Control acceleration logging
10.2.9
Query status of acceleration logging
10.2.10
Query flash statistic
10.2.11
Query boot counter
10.3
Response messages
10.3.1
Status response
10.3.2
End of data message
10.3.3
Configuration response
10.3.4
Timestamp response
10.3.5
Flash statistic response
10.3.6
boot counter response
10.4
Data message
10.4.1
12 bit data format
10.4.2
10 bit data format
10.4.3
8 bit data format
10.5
Communication example
10.6
Streaming
11
MessageObjects
11.1
return_values_from_sensor
11.2
send_msg_object
To Do
12
Sensor
12.1
Hardware
12.1.1
External Flash Memory
12.1.2
Board Redesign
12.2
Software
12.2.1
General ToDo
12.2.2
Security
13
Gateway
13.1
Hardware
13.1.1
Uninterruptible Power Supply
13.1.2
Camera
13.1.3
3g/4g/5g Module
13.1.4
Bluetooth Hardware
13.1.5
Case
13.2
Software
14
Backend
Appendix
Contributors
WS 2020 / 2021
SS 2021
WS 2021 / 2022
Commerical Applications
Literature
14.1
Merging
15
Caveates
16
DONT’T DO : feature => master
17
DO : master => feature => master
System Info
www.
BCHWTZ
.de
Big Data Case Studies
Big Data Case Studies
Benjamin Buchwitz
2022-08-28
Chapter 1
Introduction
This repo contains course materials.