Creating a tab for Big Objects is now possible using the Big Object Utility app. We created a LWC component that helps in creating a tab for Big Object. The component contains the UI from which you can view Big Object records as a list view and filter. Follow the steps below.
Step-1
Install the Big Object Utility app from AppExchange.
Step-2
Use VS Code or any other tool to create a custom LWC component. In the HTML file reference the <cloudbat-big-object-list-view>
component, the component is already a part of the managed package.
Enter the API name of the Big Object in the object-api
attribute and field API names in the fields-api-list
attribute. For example, if the Big Object API name is Archived_Contact__b check the below screenshot.

Leave .js file as it is, no changes are needed.

In the meta.xml file, enter true in the <isExposed>
tag and enter lightning__tab
in the <target>
tag.

Deploy the component to your org. Here is the Github link in case you want to refer to the example component.
Step-3
To use the Big Object Utility manage package LWC component in your component, We need to enable Lightning Web Security.
Go to Setup -> Security -> Session Setting and enable Use Lightning Web Security for Lightning web components check box.

Step-4
In this step, we create a lightning tab. Go to Setup -> User Interface -> Tabs and click on the New button.

Select the lightning component you have created in Step-1, and enter the label, name, and style. Click on the Next button. Select the visibility based on your requirement and click on the Save button.

All done, you should be able to search for the tab from the App menu.

This is how the list view should look.

Common Issues / Questions
Q1) Attempting to reference cross-namespace module cloudbat-big-object-list-view
A) Enable Lightning Web Security mentioned in Step 3.