Keeping up with Kibana: This week in Kibana for Feb 22, 2018

By
Jin Mu

Hiring

Roadmap themes updates

New platform

This week two new hires joined the new platform project and we had an official new platform team kick-off meeting. For the next two to three weeks, the team will be focused on making  modifications of the already completed platform services/components. Plugin migration plan is WIP. We also met with Kibana app team to discuss some migration tech details.

Security

The security team continued working on integrating feature controls into the saved objects management, and the infra applications. Additionally, some small modifications were made to the CSP which will be implemented in Kibana 6.7 to allow Edge to continue functioning properly.

Middleware/Alerting/Notifications/Scheduling/Secrets

With the changes to the saved objects service merged and backported the Secret Service is all ready for review. https://github.com/elastic/kibana/pull/28894

This week was spent working on the design of the actions service and vetting more details about how it will work. Some work has been done on a couple of main use cases. An RFC proposal is currently being created with a more detailed design. Will send out a follow-up when it is in a PR for review.

Saved Object

This week we finalized the development of the export API and continued working on the new import API.

Geo

Maps

Towards GA:

  • Started work on embeddability for Maps into Dashboards. Some of the groundwork and internal Kibana changes have already been completed. Please track progress here  #31137 (screenshot below displays a map in a dashboard).image4.png
  • Preliminary discussions on GeoJson upload started. #28765

Polish for 6.7/7.0 Maps Beta continues:

  • More bug fixes and UI polish as we’re getting closer to release the Maps Beta
  • Localization of code-base of the Maps-app has started. This will likely not make 6.7.0/7.0.0, but will be released with patch. #30881

Elastic Maps Service

Data production

  • Infra started work on data production of tile-data. Maintaining tile-data is a new responsibility, and will be a long term effort.
  • New layers in the pipeline. These will be published to EMS next when reviewed and validated.
    • Belgium, Belarus, Croatia, Estonia, France, Hungary, Luxembourg, Poland, Slovakia, Slovenia
    • India, Brazil
    • image6.png

7.0 Stack Upgrade

Working through resolving an issue with the upgrade assistant re-indexing of security indices (elasticsearch#39284). These indices are whitelisted, so the decision has been made to stick with the naming convention created in the 5.6 upgrade assistant (.security-X).

Browser testing coverage

Release Manager: Elastic Stack Testing

Team summaries

Canvas

The Canvas team continues to work on performance and stability in the march towards GA. We’ve long suspected that React was thrashing and killing in-browser performance, and that’s where a lot of the perceived slowness was coming from. This week we confirmed and fixed one of the largest issues, where the entire workpad, all pages, and all elements, would be re-rendered for all sorts of crazy reasons, like even moving your mouse. You can see React thrashing in this flame graph

image5.png

With a little bit of code to make things smarter about re-renders, that same graph now looks like this:image10.png

We’ve also been seeing EventEmitter memory leak message recently, and that’s now been fixed. Here’s a before and after of the memory profile:image2.pngimage1.png

But it hasn’t been all bug fixes. For example, the essql function got time_zone support this week:

image11.gif

Demodata now functions with the time filter, and the data was updated so you always have current data to play with and includes a @timestamp field so adding a time filter works from the start:image12.gif

We added content to the help menu for 7.0+, including a hotkey guide:

image3.png

Simplifying the code around the Layout Engine: Canvas is a presentation editor like Google Slides or Powerpoint. It supports lots of stateful interactions like rotation/move/resize, snap, grouping etc. going beyond regular UI, things like align/distribute and 3D planned.

Processing user interactions and rendering their results must be done fluidly, 30-60 times per second. Therefore the layout engine now updates a separate, fast, plain state object, not the Canvas Redux state with ES indices etc. attached. But maintaining and synchronizing this with the central Canvas state is error prone, hindering feature development, fixes and perf improvements. Now a silo, it's being converted to stuff all developers use, eg. React component props/state and Canvas state.

The WIP work simplifies the state management (we're targeting this for 6.7 though it's not a prerequisite):

  • no per-page layout state living in a Map outside React/Redux, having to sync with page ops
  • no longer maintaining two separate, overlapping state objects from an action stream; most layout middleware code gone and preparing for single-source of truth
  • avoids explicit rerendering triggers (setState trick)
  • simplified redux-reselect-like selectors (and selectReduce is gone)
  • helps complete the TypeScript conversion
  • adding `typings-tester` to our code base, without which TS types can degrade over time
  • due to simplifications, data-driven layout updates and an eventual fusion of tiled (Dashboard) and freeform (Canvas) editors will be easier

It becomes even more compact around 10kB - the somewhat comparable react-beautiful-dnd which would cover a small fraction of what’s needed (and things Canvas doesn’t need) is 108kB minified.

Design

The design team is finally recovering from the 7.0 sprint.  7.0 isn’t yet over for us yet though, and we’re still working through some dark mode cleanup and polish work for the Uptime app and global datepicker component. On the EUI front we finally closed our some pesky issues around focus traps and portals within modals. We’re also working towards our final CSP solution and have a bunch of fixes / additions for the range slider, filter group, and icon components. Last but not least we’re slowly making progress on converting EUI to Typescript. Phew… there’s a lot going on!

Working on some new homepage designs that lead people towards guidesimage8.png

The ACE editor now works in dark modeimage7.png

Uptime app getting some cleanupimage9.png

Operations

Our priority has been on resolving blockers with the upgrade process as they arise.

PRs

  • Improve dll plugin relation with webpackshims (#30129)
  • [logging] Corrects intercept of ECONNRESET #31742
  • [UA] Protects against Cloud plugin being disabled #31637

QA

The team focused on testing 6.7.0 release

KibanaApp (Visualizations, Sharing, Discover)

The team is still stabilizing the 7.0 release by fixing several bugs, converting remaining UI to the new design (e.g. the create new visualization dialog) and implementing minor enhancements (like copy buttons for request/responses in the Inspector or better human readable labels in the Advanced Settings).

PRs

  • Update embeddables metadata to support multiple index patterns (#31538)
  • Surface vis loader errors in the UI. (#30594)
  • tests typescript migration (#31234)
  • Add referenced pipeline aggs to every level of query (#31121)
  • Fix Visual Builder wrong color rules on multiple metric (#31455)
  • Dark overrides for Ace/Brace TM (textmate) theme (#31628)
  • Don't send request if it already failed during query parsing (#31219)
  • Add placeholder text indicating the filters agg takes only lucene and Query DSL (#31296)
  • Add visual indication of selected language in query bar (#30899)
  • TSVB error message needs to be formatted (#31569)
  • moved vis table to use EUI (#30610)
  • Add i18n context to inspector panel (#31467)
  • Use default index pattern when one is not present in visualize (#30574)
  • Remove use of 's' regex flag in KQL (#31292)
  • New visualization search selection (#30196)
  • Renamed the kuery setting to KQL (#29587)
  • Add documentation links to Visualize/Dashboard (#31406)
  • Remove geotile_grid aggregation from non-geospatial visualizations (#31354)
  • Add copy button to inspector request/response panel (#31408)
  • Add labels for sort order and request preference setting (#31236)
  • Fix bottom tooltips flickering on dashboard (#31245)
  • typescript search embedable (#30933)
  • Filter colours are not overriding label colours in Visual Builder (#30822)

Elastic Chart PRs

Cheers,
Kibana