For day 15 of the 25 Days of Rosette we asked our friends at Search Technologies to tell us about their predictions for search and big data analytics in 2017. Search Technologies, a Basis Technology partner, tackles the world’s toughest search challenges, including a recent project in which they helped a large global recruiting firm implement a system that significantly improved candidate identification, matching, and retention. From their in-the-field expertise, they give us a peek into what’s coming in search and big data. Check out their thoughts below, and don’t miss the video blog of Search Technologies’ Chief Architect Paul Nelson discussing the future of big data and search. The following is a guest blog written by the Search Technologies Marketing team.

With 2017 right around the corner, let’s take a look at some of the top search and big data trends that we predict will become more prominent and expected when it comes to search capabilities for businesses and organizations around the globe.

Real-time Personalization

Wouldn’t it be great if search results were always tailored to individual users or user groups? This might seem too good to be true, but in fact, it’s already happening (think Google when a user is logged in). We predict that personalized search results will become more regular and routine all systems.

With the help of log analytics, users can be clustered based on their past search history, download history, social media profiles, etc. For example, if a user goes to an e-commerce website to do some Christmas shopping and enters one or two searches for “Harry Potter” and “wand,” the website would group the user with other Harry Potter fans and adjust its recommendations in response.

Currently, real-time personalization is still in its infancy, but we expect to see major advances as the year goes on. User data is being captured, all of the events are flowing through and being saved in the system, but they are just being analyzed after the fact instead of in real-time.

Improved Machine Learning Algorithms

When search engines produce results for users, the results given are based on a series of algorithms that were created in the 1980s. It’s hard to believe that these algorithms are still being used today given all of the advances we’ve had in technology since. They exist because they are easy to compute inside search engines. However, computational ease doesn’t necessarily guarantee accurate results. We predict that changes are coming: Search engine algorithms will be tweaked and ad hoc algorithms will start to be replaced by statically valid machine learning ones that actually know and understand the user. This technique is known as search engine scoring.

For example, with new algorithms, a search engine can pinpoint the likelihood that a user is looking for a given document, and rank it within the results accordingly. These updated search engines can also calculate the percentage chance that a user will actually click on a returned result based on their past search history and the search history of other similar users. Machines are getting smart(er)!

The Merger of Big Data and Search

Last year, Doug Cutting, Cloudera’s Chief Architect, said: “People today think that search and big data are separate but in two or three years, everyone will wonder why we ever thought that.” In our work at Search Technologies, search and big data have always gone together, but recently they’ve merged even more closely. We’re seeing this integration in every project we’re working on: data lakes, research dashboards for research institutions, searching and matching candidates to job descriptions for the recruiting industry, preventing insider threats for government agencies, etc.

The big data revolution, as we like to call it now, has played out largely behind the scenes. However, it’s changing our lives significantly and will continue to change the way we interact with computers in profound ways.

Computers are gathering log data from all sorts of interactions and devices including cell phones, IQ codes and RFID codes. The invention of the big data parallel processing framework has made search engines even more powerful, making them even more scalable and able to handle the demands of today’s content, including real-time log analytics.

Search and big data are coming closer and closer together every day. We predict that in 2017 they will work hand in hand to do some very sophisticated things.

For a more detailed look at these predictions check out Search Technologies Chief Architect, Paul Nelson, in his new video.