Tuesday, 2 January 2018

Quick notes on SmartDB (temporary post)

Since there is a discussion going on, mostly on Twitter (link) about SmartDB, I'm going to put my own notes here. That way I have a public-place to "think out loud" about the concept of SmartDB.

Smart-DB is, in my view:
A concept where an IT system is built In the DataBase.
Constructed so that most of the work (and processing, and maintenance) 

This makes for a single point where all (99%) of the logic can be found, and where all (99%) of the code can be found and maintained.

Maximum logic/work/processing on the DB.
Minimum processing in front-end-tier (browser?).

Eliminate other layers and components.

Key point, in my view: Eliminate layers, eliminate components, and eliminate sources of problems.
Also eliminates round-trips between components, a notable source of delay.

note: SmartDB is more then just using PL/SQL or pl/pgsql, but those "stored procedure" constructs are an essential part of the SmartDB concept. Hence the focus often shifts to these tools.

Keep in mind the overall goal: 
A Working and Sustainable IT system (to process data for some end-user-business purpose).

Data and Data-model tends to survive the tools and even the "processes" of the system.

SmartDB methodology, as i  was taught years ago, would go more or less like this....

 - Ensure you have a notion of IT technology and Design. (e.g. design of process and data)
 - define the goal of your system: why will this system exist and who are the users, “consumers” ? 
 - define datamodel. what data needs to go  in+out of the system, and what needs to be stored.
 - define processes, functionality, functions.
 - refine data-model and table design. one possible Quality checks is 3NF. 
 - define the interaction between processes and data (CRUD matrix)
 - list/define the interfaces : which external actors bring/consume data to/from the system
 - define/determine the format of the interfaces (e.g. message-definitions)
 - program the interfaces to receive and return their data and statuses (e.g. error-messages)
 - check for completeness: determine the life-cycle of all entities/records.

 - logic and data in 1 place. (look no further)
 - back end can survive multiple versions of "front-end" or UX (if a front-end exists at all!)
 - troubleshooting in 1 place (look no further, no discusion, no hiding, no escape.... )
 - potential to minimize round-trips. Every "interaction" should be a message, preferably a single round trip.
 - Messages can be de-coupled (most, not all...), and queueing mechanisms can be used for resilience.
 - messages should be constructed to allow re-play (re-submit) without damage to integrity of the system  or database.
 - ACID only required in the Database. 
 - The database-replication mechanisms, and only the DB-replica!, provides DR capability.

Use/sale of Tools or Products is not a goal
Use/sale of Cloud is not a goal
Use/sale of "big data" is not a goal.
Use/sale of UX / Microservices / ClientServer / Cobol / R-sharp / Dockernetis / etc... is not a goal...

Future work:
- include links to proven methods of data-design (Heli?) . Include data-design in text.
- include links to ACID and 12+1 rules for Databases, include in text.
- include links to proven methods of Requirements and how-to Design... 
 - Do we have to start "Teaching IT from Scratch" again ??? 

So far my notes.
I also have life, you know..


Unknown said...
This comment has been removed by a blog administrator.
PdV said...

Removed previous comment, presumable just a nice-compliment (thx) to promote a link to some provider.