Data Replication With Repliers
The Repliers API is designed for real-time use, allowing your application to make requests and format responses as users interact with it. Additionally, you can replicate data using the Repliers API, provided you have the necessary data license. Be sure to check your local MLS policies on data replication before proceeding.
Data replication involves two steps: the initial download and periodic dataset refreshing with new and updated listings.
To replicate data for the last two years, start by making a request for listings (including status) and setting the updatedOn parameter to two years ago. For reference, the date this article was published is 2024-05-16.
Example Request:
GET https://api.repliers.io/listings?status=A&status=U&updatedOn=2022-05-16
The results will be paginated. The response will include the total number of listings to download (count), page number (page), results per page (pageSize), and the number of pages in the response (numPages).
You will need to paginate through the results until all listings are downloaded. Store the listings in your preferred location (e.g., database). Proceed to the next day in sequence:
Example Requests:
GET https://api.repliers.io/listings?status=A&status=U&updatedOn=2022-05-17
GET https://api.repliers.io/listings?status=A&status=U&updatedOn=2022-05-18
Continue until you reach the current date. Once completed, the initial download is finished.
You can refresh the dataset using either polling or a data stream (requires our webhooks add-on subscription). Using a data stream is more efficient, providing updates without delay. Polling may introduce delays and risk missing data due to late updates from MLSes.
Periodically make requests for new and updated listings using a timestamp.
Example Request:
GET https://api.repliers.io/listings?status=A&status=U&minUpdatedOn=2024-05-16 00:00:00
This request provides listings updated on or after May 16, 2024, at midnight. Paginate through the results and download them all. Record the start time of your job and use it as the minUpdatedOn for the next request to avoid duplicate updates.
Note: Consider overlapping the minUpdatedOn to account for late updates. Compare the updatedOn value with your records and handle only new or differently updated listings.
Subscribe to listing.created, listing.updated, and listing.deleted events using our webhooks feature. Whenever a new listing is created, an existing listing is updated, or a listing is deleted, Repliers will send a JSON payload to the URL you provide. Process this payload to store, update, or delete listings as needed.
For updated listings, the payload includes a previous object to show the changes.
For more information on Repliers Webhooks, see this link:
https://docs.repliers.io/reference/create-a-webhook
Data replication involves two steps: the initial download and periodic dataset refreshing with new and updated listings.
Step 1: Initial Download
To replicate data for the last two years, start by making a request for listings (including status) and setting the updatedOn parameter to two years ago. For reference, the date this article was published is 2024-05-16.
Example Request:
GET https://api.repliers.io/listings?status=A&status=U&updatedOn=2022-05-16
The results will be paginated. The response will include the total number of listings to download (count), page number (page), results per page (pageSize), and the number of pages in the response (numPages).
{
"page": 1,
"numPages": 5,
"pageSize": 100,
"count": 493
}You will need to paginate through the results until all listings are downloaded. Store the listings in your preferred location (e.g., database). Proceed to the next day in sequence:
Example Requests:
GET https://api.repliers.io/listings?status=A&status=U&updatedOn=2022-05-17
GET https://api.repliers.io/listings?status=A&status=U&updatedOn=2022-05-18
Continue until you reach the current date. Once completed, the initial download is finished.
Step 2: Periodically Refreshing the Dataset
You can refresh the dataset using either polling or a data stream (requires our webhooks add-on subscription). Using a data stream is more efficient, providing updates without delay. Polling may introduce delays and risk missing data due to late updates from MLSes.
Method 1: Polling
Periodically make requests for new and updated listings using a timestamp.
Example Request:
GET https://api.repliers.io/listings?status=A&status=U&minUpdatedOn=2024-05-16 00:00:00
This request provides listings updated on or after May 16, 2024, at midnight. Paginate through the results and download them all. Record the start time of your job and use it as the minUpdatedOn for the next request to avoid duplicate updates.
Note: Consider overlapping the minUpdatedOn to account for late updates. Compare the updatedOn value with your records and handle only new or differently updated listings.
Method 2: Streaming
Subscribe to listing.created, listing.updated, and listing.deleted events using our webhooks feature. Whenever a new listing is created, an existing listing is updated, or a listing is deleted, Repliers will send a JSON payload to the URL you provide. Process this payload to store, update, or delete listings as needed.
For updated listings, the payload includes a previous object to show the changes.
For more information on Repliers Webhooks, see this link:
https://docs.repliers.io/reference/create-a-webhook
Updated on: 29/11/2024
Thank you!