What’s New in Version 8?
Compared to older versions, version 8 has been designed from the ground up keeping flexibility, high performance, and integration in view. The user interface has not changed much but everything underneath it has!
Out of all the changes made, we will only cover 2 major changes in the latest version that affect deployment and configuration steps.
.NET 5 Framework
The biggest change in v8 is the fact that it now uses .NET 5 instead of a mixture of .NET Framework 4.8 and .NET Core 2.2. This meant changing all the components, including the web application as well as all the services, to either .NET 5 or .NET Standard 2.1.
With that everything internal has changed as well, including the data access layer, all methods being async etc. bringing performance boost among other things.
This also opens the possibility of deploying various application components in Linux OS as well as Containers.
Microservices using Broker
VIDIZMO always had a concept of service but with v8 we have taken it to another level. All components part of VIDIZMO application use pub/sub pattern messaging to communicate to each other. To achieve this feat, VIDIZMO uses an intermediary message broker system. This has decoupled the event (message) generation, aka publishing with event consumption, aka subscription.
Every event that is generated by API endpoint now ends up going on wire to a broker which has one or more subscribers sitting on the other end waiting to process that message. This change impacts how VIDIZMO is now deployed and configured.
Since customers may already be using Broker services of their own, VIDIZMO supplies multiple options to choose from. These are discussed later in the document.
Component Architecture
Before getting into the deployment and configuration, it is important to understand what the high-level system and component design looks like in VIDIZMO.
This section supplies details about VIDIZMO software and its components that are necessary to function for VIDIZMO to run. Here is the overview of VIDIZMO software architecture (diagram):
Web Application
It is a Single Page Application (SPA) that interacts with the Web APIs and supplies the user interface to the end users for performing various operations like uploading of content, managing users, configuring portal settings etc. This application includes a lot of small JavaScript, HTML and CSS packages that are downloaded during the bootstrap process and supply the user with all the functionalities inside the browser.
Web API
Web API exposes all the API endpoints that are used by SPA or any other interface applications. The Web API has a lot of middleware in it that a request goes through before reaching the API endpoint, including Authentication, Authorization, Tenant Context etc. which are essential in running smooth operations. A middleware dedicated for exception handling is also there which catches and logs any errors generated during the API call.
The Web API is a web application typically deployed in IIS and requires .NET framework 5 as well as .NET Core Hosting Bundle installed. .NET Core Hosting bundle installs an IIS Module called ‘ASP .NET Core Module’ that allows ASP.NET 5 applications to work with IIS.
Broker Service
The broker is the message-routing part of the system. It passes messages from client to server and from server to client. These messages are requests for services and replies to those requests, as well as messages about exceptions that have occurred. The requests are coded as calls to the broker's API. The broker handles error handling in response to these exception reports.
There are 4 available options:
- Azure Service Bus
- Kafka
- DAPR
- RabbitMQ
Azure Service Bus
Azure Service Bus is a fully managed enterprise message broker with message queues and publish-subscribe topics (in a namespace). Service Bus is used to decouple applications and services from each other, supplying the following benefits:
- Load-balancing work across competing workers
- Safely routing and transferring data and control across service and application boundaries
- Coordinating transactional work that requires a high degree of reliability
Reference: https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-messaging-overview
Kafka
Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. Reference: https://kafka.apache.org/quickstart
RabbitMQ
RabbitMQ is lightweight and easy to deploy on premises and in the cloud. It supports multiple messaging protocols. RabbitMQ can be deployed in distributed and federated configurations to meet high-scale, high-availability requirements. Reference: https://www.rabbitmq.com/how.html
DAPR
Dapr is a portable, event-driven runtime that makes it easy for any developer to build resilient, stateless and stateful applications that run on the cloud and edge and embraces the diversity of languages and developer frameworks. Leveraging the benefits of sidecar architecture, Dapr helps tackle the challenges that come with building microservices and keeps the code platform agnostic. Reference: https://docs.microsoft.com/en-us/dotnet/architecture/dapr-for-net-developers/getting-started
VIDIZMO Services
The VIDIZMO application uses different windows services as explained below:
Event Logger Service
All the events being generated because of various user or system actions are logged and stored by Event Logger Service. These logged events become the basis of Audit log as well as Chain of Custody feature in application. The data logged by the service includes the user’s information, the action that was performed as well as the data associated with that action among other things.
Workflow Service
Workflow Service is the workflow engine that powers lot of long running operations in VIDIZMO such as Transcoding, AI (Artificial Intelligence) processing, data extraction etc. Since it can take up on lot of CPU/GPU and Disk I/O intensive operations, it is best to deploy it on a separate machine that web application or rest of the services. Though all the services support scaling out, but among all of them Workflow Service is often the one that is scaled out because of the nature of its job.
Notification Service
The notification service handles all the email as well as web hook communication out of the system. Webhooks enable you to subscribe to the various events taking place within the VIDIZMO portal and receive notifications based on those events within your business application as soon as a subscribed event is occurred within the VIDIZMO Portal, it will trigger a notification payload request and send it to the specified endpoint URL.
Telemetry Service
Telemetry Service is the automatic recording and transmission of usage incurred in an Account to the database so that reports can be generated for consumption. This process includes the services used to meter usage, categories the data is divided into, the associated metrics within each category and the corresponding source of each record.
Tracking Service
Tracking service keeps track of users’ actions while playing video/audio content to figure out the engagement level of a user. Tracking service lets you know how often a media was played, for how long a user played it, how often a video was paused or resumed, gives you real-time insights and much more. All user activity on a media is calculated and displayed as Events in the Media Analytics or Quality of Experience (QoE) reporting interface of VIDIZMO.
Scheduler Service
Scheduler Service is used to manage various clock-sensitive processes by refreshing the fetched results like analytics, reports dashboard, content purge. There is an activity that runs every fifteen minutes and updates the cycle of content fetching and refreshing.
Caching Service
Caching Service supplies memory caching store to improve performance fetching data. Almost, every fetched API call ends up hitting Caching service to see if data already is cached in memory. Only if data is not available in Caching Service call go to database, in that case data is populated back in Caching Service for later retrieval.
VIDIZMO Databases
VIDIZMO application requires SQL databases to store all the meta data of the content as well as other information such as users, groups etc. While the actual content is stored as files in the storage, the meta data stored in the databases allows application to index, search as well as organize it for reporting and display purposes. VIDIZMO application uses two separate databases for it's back-end operations i.e.,main application database and notifications database.
Next steps:
Deploy VIDIZMO - V8 Prerequisites and Deployment Considerations