Read about our upcoming Code of Conduct on this issue

This instance will be upgraded to Heptapod 0.28.0 on 2022-01-18 at 14:00 UTC+1 (a few minutes of down time)

Commit c59dc59f authored by Georges Racinet's avatar Georges Racinet 🦑
Browse files

Bitbucket: inner facilities for attachment import

An attachment is described by a Bitbucket API full URL, which itself
will redirect to their AWS S3 storage, with auth done by the
query string.

Therefore we need:

- to be able to load URLs in full form
- to stop using the OAuth2 client once it's not on Bitbucket
- to load the final URL

--HG--
branch : heptapod-stable
parent 9bbf8cf6ea9c
......@@ -18,6 +18,11 @@ def issue_comments(repo, issue_id)
get_collection(path, :comment)
end
def issue_attachments(repo, issue_id)
path = "/repositories/#{repo}/issues/#{issue_id}/attachments"
get_collection(path, :attachment)
end
def pull_requests(repo)
path = "/repositories/#{repo}/pullrequests?state=ALL"
get_collection(path, :pull_request)
......
......@@ -26,6 +26,18 @@ def get(path, extra_query = {})
response.parsed
end
# Return a response for an absolute URL
#
# This is useful for parts of the Bitbucket API (e.g, attachments) that
# provide a full URL.
#
# Returns response object
def raw_get(absolute_url, follow_redirect = true)
refresh! if expired?
opts = follow_redirect ? {} : {redirect_count: connection.client.options[:max_redirects]}
connection.get(absolute_url, opts)
end
delegate :expired?, to: :connection
def refresh!
......
......@@ -13,6 +13,8 @@ class Importer
attr_reader :project, :client, :errors, :users
attr_accessor :logger
AttachmentError = Class.new(StandardError)
def initialize(project)
@project = project
@client = Bitbucket::Client.new(project.import_data.credentials)
......@@ -190,6 +192,62 @@ def import_issue_comments(issue, gitlab_issue)
end
end
# Follow redirects unless they are not on Bitbucket
#
# Requests to the Bitbucket API must be done with the OAuth2 token,
# but, requests outside of Bitbucket (typically to their AWS S3 storage)
# must not, for two reasons:
#
# - would leak the token
# - AWS S3 doesn't permit coexistence of the `Authorization` header
# and of the auth in query string that last Bitbucket redirect provides
#
# Return [external?, response or URL], either
# - [false, response]:
# response would be a non streaming response from the
# OAuth2 library (itself relying on Faraday for HTTP)
# - [true, URL]
def follow_inner_bitbucket_redirects(url)
redir_count = 0
while redir_count < 10
response = client.connection.raw_get(url, false)
case response.status
when 301, 302, 303, 307
redir_count += 1
url = response.headers['location']
Rails.logger.info("testgr: got redirection #{response.status} to #{url}")
if url.start_with?("https://bitbucket.org")
return [false, response]
else
return [true, url]
end
else
return [false, response]
end
end
raise AttachmentError.new("Too much redirection")
end
def retrieve_attachment(url, file)
is_external, url_or_response = follow_inner_bitbucket_redirects(url)
unless is_external
file.write(url_or_response.body)
return
end
HTTParty.get(url_or_response, stream_body: true) do |fragment|
case fragment.code
when 301, 302, 303, 307
next
when 200
file.write(fragment)
else
raise AttachmentError.new("Non-success status code #{fragment.code} while streaming")
end
end
end
def create_labels
LABELS.each do |label_params|
label = ::Labels::FindOrCreateService.new(nil, project, label_params).execute(skip_authorization: true)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment