reduce search results size

[schabe77]

Describe the problem you are trying to solve
I migrated from old Adwords API to new Ads API. I recognized that loading some of my reports take longer than before. First I thought it was an server issues, but then I found out that the responses are much larger. An example: when doing the request

SELECT campaign.status, ad_group.status, ad_group_criterion.status, ad_group_criterion.type
 FROM ad_group_criterion
 WHERE ad_group_criterion.negative = false
 AND campaign.status != REMOVED
 AND ad_group.status != REMOVED
 AND ad_group_criterion.status != REMOVED

on a specific account, the response stream contains 280MB of data. When I did the same request with the old Adwords Reporting API (using GZIPPED_CSV as download format) the download size was only 200KB.

Describe the solution you would like
It would be great if the response size of the Ads API could be similar to the report size of the Adwords API and not differ by a factor of 1400. The old API returned a compressed csv file, the response of the new API seems to be uncompressed. It would be great if the Ads API would communicate to the server using compressed messages.

Describe alternatives you've considered
There is not much I can do on my side, but I hope the library could

• use compressed requests/responses. As far as I could see gRPC supports gzip compression: io.grpc.stub.AbstractStub.withCompression("gzip")

Additional context
I copied the most of this ticket's content from #438 that I wrote one year ago. The old issue was closed, but only one part of the request was implemented (thanks for that again). @nwbirnie wrote that I should check back in a few months.

[jradcliff]

Could you share a recent request-id from that request that contained 280 MB of data, along with the query you used in the AdWords API? I'd like to investigate that case.

Also, are you using the new OMIT_UNSELECTED_RESOURCE_NAMES parameter? I know you ran into some issues with that parameter early on, but those issues were resolved a while back.

Thanks

[schabe77]

Hi Josh,

yes, I use the new parameter (thanks again) from the very beginning.

What I did to find some examples is to sum up SearchGoogleAdsStreamResponse.getSerializedSize() for each request. I didn't make it to 280MB, but here are three examples:

requestId=qTODgOHB8he5i0kXYfU5Rw, size=226.617.663 bytes, query=

SELECT ad_group_ad.resource_name
 FROM ad_group_ad
 WHERE campaign.status != REMOVED
 AND ad_group.status != REMOVED
 AND ad_group_ad.status != REMOVED
 AND ad_group_ad.ad.system_managed_resource_source != AD_VARIATIONS
 PARAMETERS omit_unselected_resource_names=true

requestId=R7ZPX_DmbxpDlpNlLInbXg, size=175.483.361 bytes, query=

SELECT ad_group_ad.resource_name, ad_group_ad.ad.expanded_text_ad.headline_part1, ad_group_ad.ad.expanded_text_ad.headline_part2, ad_group_ad.ad.expanded_text_ad.headline_part3, ad_group_ad.ad.expanded_text_ad.description, ad_group_ad.ad.expanded_text_ad.description2, ad_group_ad.ad.expanded_text_ad.path1, ad_group_ad.ad.expanded_text_ad.path2, ad_group_ad.ad.type, ad_group_ad.ad.final_urls, ad_group_ad.ad.url_custom_parameters
 FROM ad_group_ad
 WHERE campaign.status != REMOVED
 AND ad_group.status != REMOVED
 AND ad_group_ad.status != REMOVED
 AND ad_group_ad.ad.type = EXPANDED_TEXT_AD
 PARAMETERS omit_unselected_resource_names=true

requestId=lamT2O1-dPfI4o0P67VyeQ, size=195.195.731 bytes, query=

SELECT asset.id, asset.page_feed_asset.page_url, asset.page_feed_asset.labels
 FROM asset
 WHERE asset.type = PAGE_FEED
 PARAMETERS omit_unselected_resource_names=true

The problem is not that I get this big results sizes for a single account, the problem is, that I do the same request for hundred(s) of accounts - so even if the result is "only" 100MB large, for >100 accounts you easily run into Gigabytes.

Best regards,

Christian